Note: This is test shard 2 of 6.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from DynamicMultiMasterTest
[ RUN ] DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified
2026-08-07T07:52:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:08Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260807 07:52:08.431277 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:44633
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:44633
--master_support_change_config=false
--master_auto_join_cluster=false with env {}
W20260807 07:52:08.822908 2737 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:08.823269 2737 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:08.823424 2737 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:08.831671 2737 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:08.831758 2737 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:08.831852 2737 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:08.831948 2737 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:08.842092 2737 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:44633
--master_auto_join_cluster=false
--master_support_change_config=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:08.844183 2737 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:08.846946 2737 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:08.866183 2743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:08.866315 2742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:08.867666 2745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:08.868201 2737 server_base.cc:1061] running on GCE node
I20260807 07:52:08.869217 2737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:08.871738 2737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:08.872987 2737 hybrid_clock.cc:648] HybridClock initialized: now 1786089128872878 us; error 100 us; skew 500 ppm
I20260807 07:52:08.873572 2737 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:08.876700 2737 webserver.cc:533] Webserver started at http://127.2.169.254:36669/ using document root <none> and password file <none>
I20260807 07:52:08.877876 2737 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:08.878041 2737 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:08.878619 2737 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:08.882093 2737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0"
format_stamp: "Formatted at 2026-08-07 07:52:08 on dist-test-slave-bw88"
I20260807 07:52:08.882949 2737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0"
format_stamp: "Formatted at 2026-08-07 07:52:08 on dist-test-slave-bw88"
I20260807 07:52:08.891072 2737 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260807 07:52:08.896916 2754 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:08.899080 2737 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 07:52:08.899340 2737 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0"
format_stamp: "Formatted at 2026-08-07 07:52:08 on dist-test-slave-bw88"
I20260807 07:52:08.899608 2737 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:08.919749 2737 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:08.921057 2737 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:08.921574 2737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:08.953557 2737 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:44633
I20260807 07:52:08.953500 2821 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:44633 every 8 connection(s)
I20260807 07:52:08.956244 2737 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:08.963166 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 2737
I20260807 07:52:08.963682 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:08.966847 2823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:08.985956 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0: Bootstrap starting.
I20260807 07:52:08.992887 2823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:08.995138 2823 log.cc:826] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:08.999626 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0: No bootstrap required, opened a new log
I20260807 07:52:09.007570 2823 raft_consensus.cc:359] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } }
I20260807 07:52:09.008131 2823 raft_consensus.cc:385] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:09.008363 2823 raft_consensus.cc:740] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcedd5186ccf4c58ac7d6cbf12ad87c0, State: Initialized, Role: FOLLOWER
I20260807 07:52:09.010089 2823 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } }
I20260807 07:52:09.010553 2823 raft_consensus.cc:399] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:09.010818 2823 raft_consensus.cc:493] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:09.011119 2823 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:09.013810 2823 raft_consensus.cc:515] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } }
I20260807 07:52:09.015072 2823 leader_election.cc:304] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcedd5186ccf4c58ac7d6cbf12ad87c0; no voters:
I20260807 07:52:09.016211 2823 leader_election.cc:290] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:09.016426 2829 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:09.017280 2829 raft_consensus.cc:697] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [term 1 LEADER]: Becoming Leader. State: Replica: bcedd5186ccf4c58ac7d6cbf12ad87c0, State: Running, Role: LEADER
I20260807 07:52:09.018218 2829 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } }
I20260807 07:52:09.020004 2823 sys_catalog.cc:565] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:09.024792 2830 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } } }
I20260807 07:52:09.024718 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcedd5186ccf4c58ac7d6cbf12ad87c0. Latest consensus state: current_term: 1 leader_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 44633 } } }
I20260807 07:52:09.025297 2830 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:09.025298 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:09.030372 2841 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:09.039870 2841 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:09.055341 2841 catalog_manager.cc:1383] Generated new cluster ID: 777dfdf7aa1e4740a7f10a65738bc19c
I20260807 07:52:09.055501 2841 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:09.078909 2841 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:52:09.080487 2841 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:09.095069 2841 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bcedd5186ccf4c58ac7d6cbf12ad87c0: Generated new TSK 0
I20260807 07:52:09.096165 2841 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:09.112288 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:44633
--builtin_ntp_servers=127.2.169.212:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260807 07:52:09.481861 2854 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:09.482209 2854 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:09.482364 2854 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:09.490932 2854 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:09.491135 2854 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:09.501803 2854 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:44633
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:09.503526 2854 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:09.505697 2854 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:09.516350 2860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:09.516377 2861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:09.517524 2863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:09.518627 2854 server_base.cc:1061] running on GCE node
I20260807 07:52:09.519308 2854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:09.520341 2854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:09.521595 2854 hybrid_clock.cc:648] HybridClock initialized: now 1786089129521549 us; error 59 us; skew 500 ppm
I20260807 07:52:09.521956 2854 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:09.524384 2854 webserver.cc:533] Webserver started at http://127.2.169.193:34239/ using document root <none> and password file <none>
I20260807 07:52:09.525275 2854 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:09.525439 2854 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:09.525785 2854 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:09.528196 2854 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "2dfc92cbf41d45438c59af08fc074c81"
format_stamp: "Formatted at 2026-08-07 07:52:09 on dist-test-slave-bw88"
I20260807 07:52:09.529125 2854 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "2dfc92cbf41d45438c59af08fc074c81"
format_stamp: "Formatted at 2026-08-07 07:52:09 on dist-test-slave-bw88"
I20260807 07:52:09.534693 2854 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 07:52:09.538640 2870 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:09.539995 2854 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:09.540174 2854 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "2dfc92cbf41d45438c59af08fc074c81"
format_stamp: "Formatted at 2026-08-07 07:52:09 on dist-test-slave-bw88"
I20260807 07:52:09.540416 2854 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:09.559581 2854 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:09.560555 2854 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:09.560868 2854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:09.562234 2854 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:09.564679 2854 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:52:09.564817 2854 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:09.565057 2854 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:52:09.565214 2854 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:09.608374 2854 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:46865
I20260807 07:52:09.608480 3020 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:46865 every 8 connection(s)
I20260807 07:52:09.610121 2854 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:09.617202 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 2854
I20260807 07:52:09.617492 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:52:09.626386 3022 heartbeater.cc:344] Connected to a master server at 127.2.169.254:44633
I20260807 07:52:09.626809 3022 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:09.627709 3022 heartbeater.cc:507] Master 127.2.169.254:44633 requested a full tablet report, sending...
I20260807 07:52:09.631165 2776 ts_manager.cc:194] Registered new tserver with Master: 2dfc92cbf41d45438c59af08fc074c81 (127.2.169.193:46865)
I20260807 07:52:09.633348 2776 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:42659
I20260807 07:52:09.636512 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:09.640535 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:10.637588 3022 heartbeater.cc:499] Master 127.2.169.254:44633 was elected leader, sending a full tablet report...
I20260807 07:52:10.700124 2793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80daedd29a344d99b7c5f0fa213101bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcedd5186ccf4c58ac7d6cbf12ad87c0" is_pre_election: true
W20260807 07:52:10.724378 2757 messenger.cc:376] Unable to handle RPC call: Not implemented: call requires unsupported application feature flags: 6
I20260807 07:52:10.858808 2727 dynamic_multi_master-test.cc:523] Add master stderr: W20260807 07:52:10.014714 3027 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:10.015107 3027 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:10.015202 3027 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:10.024317 3027 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:10.024430 3027 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:10.024497 3027 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:10.024549 3027 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
Not implemented: cluster does not support AddMaster with feature(s) DYNAMIC_MULTI_MASTER
I20260807 07:52:10.863582 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 2854
I20260807 07:52:10.873508 3016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:10.962839 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 2854
I20260807 07:52:10.988767 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 2737
I20260807 07:52:10.990221 2817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:11.100788 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 2737
2026-08-07T07:52:11Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestAddMasterFeatureFlagNotSpecified (2770 ms)
[ RUN ] DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid
2026-08-07T07:52:11Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:11Z Disabled control of system clock
I20260807 07:52:11.181864 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:34137
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:34137,127.2.169.253:36753
--master_auto_join_cluster=false with env {}
W20260807 07:52:11.536684 3164 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:11.537060 3164 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:11.537222 3164 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:11.545905 3164 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:11.546008 3164 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:11.546152 3164 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:11.546234 3164 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:11.556605 3164 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:34137,127.2.169.253:36753
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:11.557003 3164 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:34137,127.2.169.253:36753'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:11.558576 3164 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:11.560724 3164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:11.570017 3170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:11.570024 3169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:11.570775 3172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:11.571574 3164 server_base.cc:1061] running on GCE node
I20260807 07:52:11.572403 3164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:11.573904 3164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:11.575163 3164 hybrid_clock.cc:648] HybridClock initialized: now 1786089131575104 us; error 31 us; skew 500 ppm
I20260807 07:52:11.575559 3164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:11.577862 3164 webserver.cc:533] Webserver started at http://127.2.169.254:41921/ using document root <none> and password file <none>
I20260807 07:52:11.578696 3164 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:11.578858 3164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:11.579278 3164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:11.581789 3164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "5f3d56ad72c1476c856279f23ba91d77"
format_stamp: "Formatted at 2026-08-07 07:52:11 on dist-test-slave-bw88"
I20260807 07:52:11.582583 3164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "5f3d56ad72c1476c856279f23ba91d77"
format_stamp: "Formatted at 2026-08-07 07:52:11 on dist-test-slave-bw88"
I20260807 07:52:11.588192 3164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 07:52:11.591626 3181 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:11.592935 3164 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 07:52:11.593119 3164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "5f3d56ad72c1476c856279f23ba91d77"
format_stamp: "Formatted at 2026-08-07 07:52:11 on dist-test-slave-bw88"
I20260807 07:52:11.593325 3164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:11.622274 3164 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:11.623173 3164 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:11.623487 3164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:11.644470 3164 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:34137
I20260807 07:52:11.644495 3243 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:34137 every 8 connection(s)
I20260807 07:52:11.646054 3164 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:11.650409 3244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:11.650647 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3164
I20260807 07:52:11.650897 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:11.652112 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:36753
--webserver_interface=127.2.169.253
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:34137,127.2.169.253:36753
--master_auto_join_cluster=false with env {}
I20260807 07:52:11.657753 3244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:11.674019 3244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:52:11.675287 3182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
W20260807 07:52:11.677143 3244 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:36753: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
I20260807 07:52:11.726636 3244 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } attempt: 1
W20260807 07:52:11.728894 3244 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:36753: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
I20260807 07:52:11.797232 3244 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } attempt: 2
W20260807 07:52:11.799486 3244 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:36753: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
I20260807 07:52:11.890808 3244 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } attempt: 3
W20260807 07:52:11.893092 3244 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:36753: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
W20260807 07:52:11.998131 3246 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:11.998485 3246 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:11.998595 3246 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:12.006958 3246 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:12.007040 3246 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:12.007090 3246 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:12.007134 3246 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:12.016675 3246 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:34137,127.2.169.253:36753
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:36753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:12.016978 3246 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:34137,127.2.169.253:36753'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:12.018291 3246 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:12.020087 3246 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:12.028051 3254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:12.028031 3255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:12.028884 3257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:12.029585 3246 server_base.cc:1061] running on GCE node
I20260807 07:52:12.030382 3246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:12.031718 3246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:12.032964 3246 hybrid_clock.cc:648] HybridClock initialized: now 1786089132032902 us; error 45 us; skew 500 ppm
I20260807 07:52:12.033300 3246 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:12.035472 3246 webserver.cc:533] Webserver started at http://127.2.169.253:38129/ using document root <none> and password file <none>
I20260807 07:52:12.036196 3246 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:12.036357 3246 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:12.036392 3244 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } attempt: 4
I20260807 07:52:12.036772 3246 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260807 07:52:12.038393 3244 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:36753: Network error: Client connection negotiation failed: client connection to 127.2.169.253:36753: connect: Connection refused (error 111)
I20260807 07:52:12.039521 3246 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "b17ba3af19654b5eb24897fb40849975"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.040297 3246 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "b17ba3af19654b5eb24897fb40849975"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.045531 3246 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 07:52:12.048777 3264 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:12.050259 3246 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:12.050451 3246 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "b17ba3af19654b5eb24897fb40849975"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.050661 3246 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:12.071774 3246 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:12.072818 3246 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:12.073158 3246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:12.095247 3331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:36753 every 8 connection(s)
I20260807 07:52:12.095269 3246 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:36753
I20260807 07:52:12.096947 3246 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:12.101605 3332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:12.103955 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3246
I20260807 07:52:12.104139 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:52:12.109596 3332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:12.124500 3332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:12.134684 3332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: Bootstrap starting.
I20260807 07:52:12.138185 3332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:12.139474 3332 log.cc:826] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:12.142140 3332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: No bootstrap required, opened a new log
I20260807 07:52:12.147097 3332 raft_consensus.cc:359] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.147398 3332 raft_consensus.cc:385] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:12.147509 3332 raft_consensus.cc:740] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17ba3af19654b5eb24897fb40849975, State: Initialized, Role: FOLLOWER
I20260807 07:52:12.148129 3332 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.149111 3341 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.149484 3341 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:12.150262 3332 sys_catalog.cc:565] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:12.159823 3357 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:12.159931 3357 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:12.169498 3341 raft_consensus.cc:493] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:12.169750 3341 raft_consensus.cc:515] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.171008 3341 leader_election.cc:290] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3d56ad72c1476c856279f23ba91d77 (127.2.169.254:34137)
W20260807 07:52:12.325445 3272 tablet.cc:2406] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:12.337633 3244 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } attempt: 5
I20260807 07:52:12.350378 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: Bootstrap starting.
I20260807 07:52:12.354842 3244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:12.356278 3244 log.cc:826] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:12.360396 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: No bootstrap required, opened a new log
I20260807 07:52:12.368309 3244 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.368673 3244 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:12.368808 3244 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3d56ad72c1476c856279f23ba91d77, State: Initialized, Role: FOLLOWER
I20260807 07:52:12.369567 3244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.370599 3360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.370909 3360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:12.371837 3244 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:12.371994 3216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b17ba3af19654b5eb24897fb40849975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3d56ad72c1476c856279f23ba91d77" is_pre_election: true
I20260807 07:52:12.372581 3216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17ba3af19654b5eb24897fb40849975 in term 0.
I20260807 07:52:12.373292 3268 leader_election.cc:304] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 5f3d56ad72c1476c856279f23ba91d77, b17ba3af19654b5eb24897fb40849975; no voters:
I20260807 07:52:12.374117 3341 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 07:52:12.374287 3341 raft_consensus.cc:493] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:12.374401 3341 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 0 FOLLOWER]: Advancing to term 1
W20260807 07:52:12.376051 3189 tablet.cc:2406] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:12.377173 3341 raft_consensus.cc:515] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.377801 3341 leader_election.cc:290] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3d56ad72c1476c856279f23ba91d77 (127.2.169.254:34137)
I20260807 07:52:12.378454 3216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b17ba3af19654b5eb24897fb40849975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3d56ad72c1476c856279f23ba91d77"
I20260807 07:52:12.378647 3216 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:12.381429 3216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17ba3af19654b5eb24897fb40849975 in term 1.
I20260807 07:52:12.382489 3268 leader_election.cc:304] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 5f3d56ad72c1476c856279f23ba91d77, b17ba3af19654b5eb24897fb40849975; no voters:
I20260807 07:52:12.382897 3341 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:12.383463 3341 raft_consensus.cc:697] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [term 1 LEADER]: Becoming Leader. State: Replica: b17ba3af19654b5eb24897fb40849975, State: Running, Role: LEADER
I20260807 07:52:12.383989 3341 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } }
I20260807 07:52:12.387337 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17ba3af19654b5eb24897fb40849975. Latest consensus state: current_term: 1 leader_uuid: "b17ba3af19654b5eb24897fb40849975" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.387585 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:12.388645 3369 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:12.391150 3369 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:12.400187 3216 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [term 1 FOLLOWER]: Refusing update from remote peer b17ba3af19654b5eb24897fb40849975: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:52:12.401832 3365 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 07:52:12.404109 3380 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:12.404239 3380 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:12.413656 3360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17ba3af19654b5eb24897fb40849975. Latest consensus state: current_term: 1 leader_uuid: "b17ba3af19654b5eb24897fb40849975" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.413847 3360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:12.416218 3341 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b17ba3af19654b5eb24897fb40849975" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.416513 3341 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:12.418678 3369 catalog_manager.cc:1383] Generated new cluster ID: 6162e6d63abe45f4afd6c57d546212ee
I20260807 07:52:12.418825 3369 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:12.426899 3360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b17ba3af19654b5eb24897fb40849975" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3d56ad72c1476c856279f23ba91d77" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 34137 } } peers { permanent_uuid: "b17ba3af19654b5eb24897fb40849975" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 36753 } } }
I20260807 07:52:12.427094 3360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:12.444525 3369 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:52:12.446099 3369 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:12.462700 3369 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b17ba3af19654b5eb24897fb40849975: Generated new TSK 0
I20260807 07:52:12.463973 3369 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:12.470189 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:34137,127.2.169.253:36753
--builtin_ntp_servers=127.2.169.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260807 07:52:12.840086 3386 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:12.840430 3386 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:12.840564 3386 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:12.849624 3386 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:12.849809 3386 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:12.860199 3386 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:34137,127.2.169.253:36753
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:12.861932 3386 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:12.863883 3386 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:12.873278 3392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:12.874066 3391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:12.875561 3394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:12.878816 3386 server_base.cc:1061] running on GCE node
I20260807 07:52:12.879452 3386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:12.880506 3386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:12.881726 3386 hybrid_clock.cc:648] HybridClock initialized: now 1786089132881667 us; error 49 us; skew 500 ppm
I20260807 07:52:12.882074 3386 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:12.884603 3386 webserver.cc:533] Webserver started at http://127.2.169.193:38291/ using document root <none> and password file <none>
I20260807 07:52:12.885726 3386 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:12.885852 3386 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:12.886309 3386 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:12.889449 3386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "d129afdf2af8412480bcab778fc99a2c"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.890475 3386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "d129afdf2af8412480bcab778fc99a2c"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.897408 3386 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 07:52:12.901500 3401 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:12.902896 3386 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:12.903074 3386 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "d129afdf2af8412480bcab778fc99a2c"
format_stamp: "Formatted at 2026-08-07 07:52:12 on dist-test-slave-bw88"
I20260807 07:52:12.903286 3386 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:12.923859 3386 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:12.924741 3386 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:12.925030 3386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:12.926103 3386 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:12.927870 3386 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:52:12.927975 3386 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:12.928089 3386 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:52:12.928166 3386 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:12.963878 3386 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:42463
I20260807 07:52:12.963941 3547 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:42463 every 8 connection(s)
I20260807 07:52:12.965420 3386 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:12.968740 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3386
I20260807 07:52:12.968988 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:52:12.986140 3549 heartbeater.cc:344] Connected to a master server at 127.2.169.254:34137
I20260807 07:52:12.986573 3549 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:12.987511 3549 heartbeater.cc:507] Master 127.2.169.254:34137 requested a full tablet report, sending...
I20260807 07:52:12.988257 3548 heartbeater.cc:344] Connected to a master server at 127.2.169.253:36753
I20260807 07:52:12.988400 3548 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:12.988952 3548 heartbeater.cc:507] Master 127.2.169.253:36753 requested a full tablet report, sending...
I20260807 07:52:12.990410 3203 ts_manager.cc:194] Registered new tserver with Master: d129afdf2af8412480bcab778fc99a2c (127.2.169.193:42463)
I20260807 07:52:12.990751 3289 ts_manager.cc:194] Registered new tserver with Master: d129afdf2af8412480bcab778fc99a2c (127.2.169.193:42463)
I20260807 07:52:12.992784 3289 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:60841
I20260807 07:52:12.994304 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:13.000694 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
W20260807 07:52:13.009297 3237 generic_service.cc:139] {username='slave'} at 127.0.0.1:39832 forcing change of non-runtime-safe flag leader_failure_max_missed_heartbeat_periods
I20260807 07:52:13.009716 3237 generic_service.cc:175] {username='slave'} at 127.0.0.1:39832 changed flags via RPC: leader_failure_max_missed_heartbeat_periods from '3' to '10.0'
W20260807 07:52:13.011322 3326 generic_service.cc:139] {username='slave'} at 127.0.0.1:42232 forcing change of non-runtime-safe flag leader_failure_max_missed_heartbeat_periods
I20260807 07:52:13.011473 3326 generic_service.cc:175] {username='slave'} at 127.0.0.1:42232 changed flags via RPC: leader_failure_max_missed_heartbeat_periods from '3' to '10.0'
I20260807 07:52:13.406488 3380 catalog_manager.cc:1295] Loaded cluster ID: 6162e6d63abe45f4afd6c57d546212ee
I20260807 07:52:13.406651 3380 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: loading cluster ID for follower catalog manager: success
I20260807 07:52:13.410374 3380 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: acquiring CA information for follower catalog manager: success
I20260807 07:52:13.412318 3380 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5f3d56ad72c1476c856279f23ba91d77: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
E20260807 07:52:13.421296 3289 master_service.cc:350] Failed removing master 127.2.169.254:34137. Invalid argument: Mismatch in UUID of the master 127.2.169.254:34137 to be removed. Expected: 5f3d56ad72c1476c856279f23ba91d77, supplied: 7792540387893288580.
I20260807 07:52:13.521124 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3386
I20260807 07:52:13.530278 3542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:13.652859 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3386
I20260807 07:52:13.680140 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3164
I20260807 07:52:13.681429 3237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:13.787719 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3164
I20260807 07:52:13.818297 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3246
I20260807 07:52:13.819564 3326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:13.927078 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3246
2026-08-07T07:52:13Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestRemoveMasterMismatchHostnameAndUuid (2825 ms)
[----------] 2 tests from DynamicMultiMasterTest (5596 ms total)
[----------] 1 test from AutoAddMasterTest
[ RUN ] AutoAddMasterTest.TestAddWithOnGoingDdl
2026-08-07T07:52:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:13Z Disabled control of system clock
I20260807 07:52:14.008127 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589 with env {}
W20260807 07:52:14.358372 3583 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:14.358690 3583 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:14.358770 3583 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:14.367223 3583 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:14.367311 3583 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:14.367367 3583 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:14.367412 3583 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:14.377805 3583 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:14.378201 3583 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:45309,127.2.169.253:45589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:14.379654 3583 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:14.381660 3583 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:14.390281 3591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:14.390667 3593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:14.391248 3595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:14.391919 3583 server_base.cc:1061] running on GCE node
I20260807 07:52:14.392745 3583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:14.394161 3583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:14.395414 3583 hybrid_clock.cc:648] HybridClock initialized: now 1786089134395347 us; error 50 us; skew 500 ppm
I20260807 07:52:14.395771 3583 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:14.398113 3583 webserver.cc:533] Webserver started at http://127.2.169.254:40051/ using document root <none> and password file <none>
I20260807 07:52:14.398864 3583 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:14.399019 3583 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:14.399382 3583 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:14.401912 3583 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.402719 3583 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.408262 3583 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 07:52:14.411780 3601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:14.413246 3583 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 07:52:14.413542 3583 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.413767 3583 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:14.448689 3583 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:14.449663 3583 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:14.449985 3583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:14.471896 3583 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:45309
I20260807 07:52:14.471982 3665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:45309 every 8 connection(s)
I20260807 07:52:14.473675 3583 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:14.478129 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3583
I20260807 07:52:14.478367 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:14.478451 3666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:14.480127 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--webserver_interface=127.2.169.253
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589 with env {}
I20260807 07:52:14.487843 3666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:14.512898 3666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:52:14.514331 3604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
W20260807 07:52:14.516263 3666 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
I20260807 07:52:14.565691 3666 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } attempt: 1
W20260807 07:52:14.568276 3666 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
I20260807 07:52:14.636564 3666 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } attempt: 2
W20260807 07:52:14.638813 3666 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
I20260807 07:52:14.730100 3666 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } attempt: 3
W20260807 07:52:14.732354 3666 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
W20260807 07:52:14.850344 3668 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:14.850709 3668 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:14.850854 3668 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:14.859370 3668 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:14.859467 3668 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:14.859557 3668 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:14.859643 3668 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:14.869808 3668 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:14.870204 3668 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:45309,127.2.169.253:45589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:14.871678 3668 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:14.873618 3668 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260807 07:52:14.875622 3666 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } attempt: 4
W20260807 07:52:14.877946 3666 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111)
W20260807 07:52:14.882725 3677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:14.882925 3680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:14.883383 3688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:14.883839 3668 server_base.cc:1061] running on GCE node
I20260807 07:52:14.884696 3668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:14.886308 3668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:14.887599 3668 hybrid_clock.cc:648] HybridClock initialized: now 1786089134887546 us; error 72 us; skew 500 ppm
I20260807 07:52:14.887993 3668 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:14.890530 3668 webserver.cc:533] Webserver started at http://127.2.169.253:41101/ using document root <none> and password file <none>
I20260807 07:52:14.891338 3668 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:14.891502 3668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:14.891904 3668 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:14.894742 3668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.895583 3668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.901643 3668 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 07:52:14.905320 3694 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:14.906840 3668 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:14.907030 3668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:14.907230 3668 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:14.920058 3668 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:14.920849 3668 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:14.921160 3668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:14.941913 3668 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:45589
I20260807 07:52:14.941941 3755 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:45589 every 8 connection(s)
I20260807 07:52:14.943503 3668 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:14.946990 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3668
I20260807 07:52:14.947245 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:52:14.948491 3756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:14.955907 3756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:14.968288 3756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:14.978242 3756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap starting.
I20260807 07:52:14.981606 3756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:14.982622 3756 log.cc:826] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:14.985165 3756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: No bootstrap required, opened a new log
I20260807 07:52:14.990619 3756 raft_consensus.cc:359] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:14.990902 3756 raft_consensus.cc:385] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:14.990979 3756 raft_consensus.cc:740] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a486fca3ed6e467daa9af0e316f2e075, State: Initialized, Role: FOLLOWER
I20260807 07:52:14.991537 3756 consensus_queue.cc:260] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:14.992307 3764 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:14.992583 3764 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:14.993398 3756 sys_catalog.cc:565] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:15.003700 3777 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:15.003841 3777 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:15.000603 3668 master_runner.cc:187] Error getting master registration for 127.2.169.254:45309: OK, instance_id { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" instance_seqno: 1786089134466078 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 07:52:15.007251 3668 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260807 07:52:15.141222 3764 raft_consensus.cc:493] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:15.141597 3764 raft_consensus.cc:515] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:15.143270 3764 leader_election.cc:290] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca5ccffd14db4e15be5ec8b65fc2502a (127.2.169.254:45309)
W20260807 07:52:15.173429 3701 tablet.cc:2406] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:15.177187 3666 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } attempt: 5
I20260807 07:52:15.188975 3666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap starting.
I20260807 07:52:15.192536 3666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:15.193958 3666 log.cc:826] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:15.197098 3666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: No bootstrap required, opened a new log
I20260807 07:52:15.202404 3666 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:15.202736 3666 raft_consensus.cc:385] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:15.202843 3666 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Initialized, Role: FOLLOWER
I20260807 07:52:15.203547 3666 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:15.204597 3781 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:15.204890 3781 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:15.205840 3666 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:15.205997 3635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a486fca3ed6e467daa9af0e316f2e075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" is_pre_election: true
I20260807 07:52:15.206650 3635 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a486fca3ed6e467daa9af0e316f2e075 in term 0.
I20260807 07:52:15.207650 3697 leader_election.cc:304] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:15.208365 3764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 07:52:15.208503 3764 raft_consensus.cc:493] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:15.208595 3764 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:15.211756 3764 raft_consensus.cc:515] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:15.212891 3764 leader_election.cc:290] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [CANDIDATE]: Term 1 election: Requested vote from peers ca5ccffd14db4e15be5ec8b65fc2502a (127.2.169.254:45309)
I20260807 07:52:15.213016 3634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a486fca3ed6e467daa9af0e316f2e075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
I20260807 07:52:15.213315 3634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:15.218032 3634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a486fca3ed6e467daa9af0e316f2e075 in term 1.
I20260807 07:52:15.218549 3697 leader_election.cc:304] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:15.218914 3764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:15.226387 3764 raft_consensus.cc:697] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 LEADER]: Becoming Leader. State: Replica: a486fca3ed6e467daa9af0e316f2e075, State: Running, Role: LEADER
I20260807 07:52:15.226903 3764 consensus_queue.cc:237] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } }
I20260807 07:52:15.229861 3790 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a486fca3ed6e467daa9af0e316f2e075. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:15.230186 3790 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:15.240069 3798 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260807 07:52:15.242451 3795 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 07:52:15.242502 3798 catalog_manager.cc:1520] Initializing Kudu cluster ID...
W20260807 07:52:15.242581 3795 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:15.252712 3634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Refusing update from remote peer a486fca3ed6e467daa9af0e316f2e075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:52:15.253875 3790 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 07:52:15.267189 3781 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a486fca3ed6e467daa9af0e316f2e075. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:15.267414 3781 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:15.273279 3802 mvcc.cc:204] Tried to move back new op lower bound from 7315821097980276736 to 7315821097893273600. Current Snapshot: MvccSnapshot[applied={T|T < 7315821097980276736}]
I20260807 07:52:15.277817 3790 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:15.278080 3790 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:15.280784 3798 catalog_manager.cc:1383] Generated new cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:15.280896 3798 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:15.291925 3781 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } }
I20260807 07:52:15.292129 3781 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:15.326753 3798 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:52:15.328756 3798 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:15.347400 3798 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Generated new TSK 0
I20260807 07:52:15.348605 3798 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:15.375250 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260807 07:52:15.783342 3807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:15.783687 3807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:15.783814 3807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:15.792296 3807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:15.792516 3807 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:15.803138 3807 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:15.804977 3807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:15.807076 3807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:15.816857 3816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:15.816879 3817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:15.817983 3822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:15.818615 3807 server_base.cc:1061] running on GCE node
I20260807 07:52:15.819269 3807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:15.820358 3807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:15.821548 3807 hybrid_clock.cc:648] HybridClock initialized: now 1786089135821480 us; error 60 us; skew 500 ppm
I20260807 07:52:15.821869 3807 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:15.824003 3807 webserver.cc:533] Webserver started at http://127.2.169.193:34149/ using document root <none> and password file <none>
I20260807 07:52:15.824803 3807 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:15.824930 3807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:15.825307 3807 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:15.828001 3807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:15.828799 3807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:15.834002 3807 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 07:52:15.837463 3828 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:15.838825 3807 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:15.838999 3807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:15.839197 3807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:15.858459 3807 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:15.859392 3807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:15.859675 3807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:15.861039 3807 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:15.862861 3807 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:52:15.862982 3807 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:15.863083 3807 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:52:15.863180 3807 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:15.900023 3807 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:36591
I20260807 07:52:15.900131 3980 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:36591 every 8 connection(s)
I20260807 07:52:15.901610 3807 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:15.911616 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 3807
I20260807 07:52:15.911845 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:52:15.917505 3982 heartbeater.cc:344] Connected to a master server at 127.2.169.254:45309
I20260807 07:52:15.917657 3981 heartbeater.cc:344] Connected to a master server at 127.2.169.253:45589
I20260807 07:52:15.917951 3982 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:15.917963 3981 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:15.919070 3982 heartbeater.cc:507] Master 127.2.169.254:45309 requested a full tablet report, sending...
I20260807 07:52:15.919135 3981 heartbeater.cc:507] Master 127.2.169.253:45589 requested a full tablet report, sending...
I20260807 07:52:15.921408 3714 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:15.921422 3623 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:15.923048 3714 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:56979
I20260807 07:52:15.932799 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:15.937938 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:15.965647 3714 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41360:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:15.997813 3891 tablet_service.cc:1511] Processing CreateTablet for tablet 7c30a1b53ff54e9a8515149bd3d5d84f (DEFAULT_TABLE table=test-workload [id=e956c0fa574044c6b86ec65f8705a989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:15.999256 3891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c30a1b53ff54e9a8515149bd3d5d84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:16.010089 4011 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:16.013463 4011 tablet_bootstrap.cc:654] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:16.014883 4011 log.cc:826] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:16.017433 4011 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:16.017724 4011 ts_tablet_manager.cc:1403] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260807 07:52:16.023234 4011 raft_consensus.cc:359] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.023537 4011 raft_consensus.cc:385] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:16.023648 4011 raft_consensus.cc:740] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:16.024271 4011 consensus_queue.cc:260] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.024487 4011 raft_consensus.cc:399] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:16.024628 4011 raft_consensus.cc:493] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:16.024803 4011 raft_consensus.cc:3060] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:16.026968 4011 raft_consensus.cc:515] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.027455 4011 leader_election.cc:304] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:16.028149 4011 leader_election.cc:290] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:16.028287 4013 raft_consensus.cc:2804] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:16.028911 4013 raft_consensus.cc:697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:16.029703 3981 heartbeater.cc:499] Master 127.2.169.253:45589 was elected leader, sending a full tablet report...
I20260807 07:52:16.029703 4013 consensus_queue.cc:237] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.030396 4011 ts_tablet_manager.cc:1434] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260807 07:52:16.037563 3714 catalog_manager.cc:5697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:16.050683 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--webserver_interface=127.2.169.252
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947 with env {}
I20260807 07:52:16.103585 3714 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41364:
name: "default.0_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:16.104157 3713 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41378:
name: "default.1_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:16.116133 3891 tablet_service.cc:1511] Processing CreateTablet for tablet fda7602334414a979800d942a24cfb86 (DEFAULT_TABLE table=default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:16.116624 3891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda7602334414a979800d942a24cfb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:16.121327 3890 tablet_service.cc:1511] Processing CreateTablet for tablet 23de065eeead41b8aaec4866731369f5 (DEFAULT_TABLE table=default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:16.122114 3890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23de065eeead41b8aaec4866731369f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:16.124877 4011 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:16.126803 4011 tablet_bootstrap.cc:654] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:16.137022 4011 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:16.137200 4011 ts_tablet_manager.cc:1403] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260807 07:52:16.138140 4011 raft_consensus.cc:359] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.138338 4011 raft_consensus.cc:385] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:16.138466 4011 raft_consensus.cc:740] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:16.138741 4011 consensus_queue.cc:260] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.138908 4011 raft_consensus.cc:399] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:16.139002 4011 raft_consensus.cc:493] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:16.139107 4011 raft_consensus.cc:3060] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:16.141398 4011 raft_consensus.cc:515] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.141664 4011 leader_election.cc:304] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:16.141860 4011 leader_election.cc:290] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:16.141932 4013 raft_consensus.cc:2804] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:16.142441 4013 raft_consensus.cc:697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:16.142678 4013 consensus_queue.cc:237] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.143790 4011 ts_tablet_manager.cc:1434] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:52:16.144083 4011 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:16.145452 3713 catalog_manager.cc:5697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:16.147361 4011 tablet_bootstrap.cc:654] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:16.155776 4011 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:16.156038 4011 ts_tablet_manager.cc:1403] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260807 07:52:16.156838 4011 raft_consensus.cc:359] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.157059 4011 raft_consensus.cc:385] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:16.157145 4011 raft_consensus.cc:740] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:16.157433 4011 consensus_queue.cc:260] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.157701 4011 raft_consensus.cc:399] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:16.157832 4011 raft_consensus.cc:493] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:16.157945 4011 raft_consensus.cc:3060] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:16.161432 4011 raft_consensus.cc:515] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.161717 4011 leader_election.cc:304] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:16.161916 4011 leader_election.cc:290] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:16.162395 4011 ts_tablet_manager.cc:1434] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:52:16.162447 4013 raft_consensus.cc:2804] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:16.162648 4013 raft_consensus.cc:697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:16.162928 4013 consensus_queue.cc:237] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:16.166630 3713 catalog_manager.cc:5697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:16.244655 3795 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:16.244781 3795 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: loading cluster ID for follower catalog manager: success
I20260807 07:52:16.247506 3795 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: acquiring CA information for follower catalog manager: success
I20260807 07:52:16.249356 3795 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 07:52:16.452198 4025 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:16.452524 4025 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:16.452666 4025 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:16.461280 4025 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:16.461386 4025 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:16.461516 4025 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:16.461589 4025 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:16.472039 4025 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:16.473906 4025 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:16.475878 4025 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:16.485162 4051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:16.486414 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:16.486785 4054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:16.487855 4025 server_base.cc:1061] running on GCE node
I20260807 07:52:16.488651 4025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:16.490204 4025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:16.491434 4025 hybrid_clock.cc:648] HybridClock initialized: now 1786089136491379 us; error 68 us; skew 500 ppm
I20260807 07:52:16.491806 4025 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:16.494405 4025 webserver.cc:533] Webserver started at http://127.2.169.252:46357/ using document root <none> and password file <none>
I20260807 07:52:16.495514 4025 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:16.495656 4025 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:16.496169 4025 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:16.499333 4025 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/instance:
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:16.500243 4025 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal/instance:
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:16.506309 4025 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 07:52:16.510066 4060 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:16.511569 4025 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:16.511790 4025 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:16.512053 4025 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:16.533222 4025 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:16.534248 4025 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:16.534534 4025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:16.557282 4025 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:39947
I20260807 07:52:16.557310 4134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:39947 every 8 connection(s)
I20260807 07:52:16.558880 4025 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:16.563390 4137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:16.563974 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4025
I20260807 07:52:16.572134 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:16.582612 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:16.588647 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:16.598690 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap starting.
I20260807 07:52:16.602103 4137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:16.603245 4137 log.cc:826] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:16.605878 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: No bootstrap required, opened a new log
I20260807 07:52:16.611069 4137 raft_consensus.cc:359] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } }
I20260807 07:52:16.611361 4137 raft_consensus.cc:385] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:16.611511 4137 raft_consensus.cc:740] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e50b67caa47fb95d0c1e1beefd731, State: Initialized, Role: FOLLOWER
I20260807 07:52:16.612187 4137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } }
I20260807 07:52:16.613142 4145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } }
I20260807 07:52:16.613445 4145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:16.614125 4137 sys_catalog.cc:565] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:16.624745 4161 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:16.624888 4161 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:16.629364 4025 master_runner.cc:429] Detected that this master 279e50b67caa47fb95d0c1e1beefd731 is joining an existing cluster
I20260807 07:52:16.629477 4025 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260807 07:52:16.630604 4025 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:52:16.631076 4025 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 07:52:16.631306 4025 tablet_replica.cc:333] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: stopping tablet replica
I20260807 07:52:16.633513 4025 master_runner.cc:434] Initiating AddMaster RPC to add 127.2.169.252:39947
I20260807 07:52:16.665501 3713 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.2.169.252:39947: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731"
member_type: NON_VOTER
last_known_addr {
host: "127.2.169.252"
port: 39947
}
attrs {
promote: true
}
}
dest_uuid: "a486fca3ed6e467daa9af0e316f2e075"
cas_config_opid_index: -1
I20260807 07:52:16.667645 3713 consensus_queue.cc:237] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } }
I20260807 07:52:16.670516 3634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Refusing update from remote peer a486fca3ed6e467daa9af0e316f2e075: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20260807 07:52:16.671247 3843 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260807 07:52:16.671660 3698 consensus_peers.cc:597] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 -> Peer 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947): Couldn't send request to peer 279e50b67caa47fb95d0c1e1beefd731. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:16.674309 4173 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.675084 3634 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index -1 to 15, NON_VOTER 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.677557 4173 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.2.169.252:39947
I20260807 07:52:16.679809 4025 master.cc:562] Master@127.2.169.252:39947 shutting down...
I20260807 07:52:16.691660 4175 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.691963 4175 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:16.705053 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.705462 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:16.706161 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.706375 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:16.725059 4025 master.cc:584] Master@127.2.169.252:39947 shutdown complete.
I20260807 07:52:16.725214 4025 master_runner.cc:306] Clearing existing system tablet
I20260807 07:52:16.725744 4025 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:16.725876 4025 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:16.728857 4025 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 07:52:16.730643 4179 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:16.731033 4025 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:16.731307 4025 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 07:52:16.731426 4025 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:16.736726 4025 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 07:52:16.744138 4025 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 07:52:16.744297 4025 log.cc:1199] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Deleting WAL directory at /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 07:52:16.744895 4025 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Deleting consensus metadata
I20260807 07:52:16.745187 4025 master_runner.cc:316] Copying system tablet from 127.2.169.253:45589
I20260807 07:52:16.748239 4025 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: tablet copy: Beginning tablet copy session from remote peer at address 127.2.169.253:45589
I20260807 07:52:16.756683 3739 tablet_copy_service.cc:140] P a486fca3ed6e467daa9af0e316f2e075: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 279e50b67caa47fb95d0c1e1beefd731 ({username='slave'} at 127.0.0.1:41400)
I20260807 07:52:16.757066 3739 tablet_copy_service.cc:161] P a486fca3ed6e467daa9af0e316f2e075: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 279e50b67caa47fb95d0c1e1beefd731 at {username='slave'} at 127.0.0.1:41400: session id = 279e50b67caa47fb95d0c1e1beefd731-00000000000000000000000000000000
I20260807 07:52:16.760365 3739 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 07:52:16.762869 4025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 07:52:16.770303 4025 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: tablet copy: Starting download of 0 data blocks...
I20260807 07:52:16.770665 4025 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: tablet copy: Starting download of 1 WAL segments...
I20260807 07:52:16.773976 4025 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 07:52:16.778708 3739 tablet_copy_service.cc:342] P a486fca3ed6e467daa9af0e316f2e075: Request end of tablet copy session 279e50b67caa47fb95d0c1e1beefd731-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:41400
I20260807 07:52:16.779001 3739 tablet_copy_service.cc:434] P a486fca3ed6e467daa9af0e316f2e075: ending tablet copy session 279e50b67caa47fb95d0c1e1beefd731-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 279e50b67caa47fb95d0c1e1beefd731
W20260807 07:52:16.786633 4025 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260807 07:52:16.790967 4025 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:16.793572 4188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:16.793635 4189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:16.794872 4192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:16.795146 4025 server_base.cc:1061] running on GCE node
I20260807 07:52:16.795475 4025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:16.796075 4025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:16.797289 4025 hybrid_clock.cc:648] HybridClock initialized: now 1786089136797217 us; error 38 us; skew 500 ppm
I20260807 07:52:16.797528 4025 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:16.798836 4025 webserver.cc:533] Webserver started at http://127.2.169.252:43361/ using document root <none> and password file <none>
I20260807 07:52:16.799260 4025 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:16.799364 4025 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:16.802361 4025 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 07:52:16.804294 4199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:16.804777 4025 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 07:52:16.804898 4025 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:16.805074 4025 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:16.823216 4025 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:16.824036 4025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:16.844355 4025 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:39947
I20260807 07:52:16.844380 4266 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:39947 every 8 connection(s)
I20260807 07:52:16.845515 4025 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:16.849045 4267 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:16.851621 4267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap starting.
I20260807 07:52:16.877065 4267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=9 ignored=0} mutations{seen=7 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:16.877652 4267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap complete.
I20260807 07:52:16.878968 4267 raft_consensus.cc:359] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } }
I20260807 07:52:16.879233 4267 raft_consensus.cc:740] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 279e50b67caa47fb95d0c1e1beefd731, State: Initialized, Role: LEARNER
I20260807 07:52:16.879514 4267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } }
I20260807 07:52:16.880421 4273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:16.881049 4267 sys_catalog.cc:565] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:16.881049 4273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:16.899264 4288 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:16.899459 4288 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: loading cluster ID for follower catalog manager: success
I20260807 07:52:16.905771 4288 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: acquiring CA information for follower catalog manager: success
I20260807 07:52:16.910353 4288 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:17.199333 3713 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41408:
name: "default.0_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:17.237248 3890 tablet_service.cc:1511] Processing CreateTablet for tablet de31825f9a984012abe2d537ea2b84f5 (DEFAULT_TABLE table=default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:17.237838 3890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de31825f9a984012abe2d537ea2b84f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:17.243949 3713 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41414:
name: "default.1_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:17.253222 4314 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:17.257030 4314 tablet_bootstrap.cc:654] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:17.261827 4314 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:17.262018 4314 ts_tablet_manager.cc:1403] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 07:52:17.263725 3890 tablet_service.cc:1511] Processing CreateTablet for tablet ba7a33d575f4453dbf1b8861634a3f98 (DEFAULT_TABLE table=default.1_2 [id=44b956d4571840a780d714244fff640c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:17.264266 3890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba7a33d575f4453dbf1b8861634a3f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:17.265990 4314 raft_consensus.cc:359] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.266337 4314 raft_consensus.cc:385] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:17.266549 4314 raft_consensus.cc:740] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:17.266914 4314 consensus_queue.cc:260] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.267215 4314 raft_consensus.cc:399] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:17.268067 4314 raft_consensus.cc:493] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:17.268357 4314 raft_consensus.cc:3060] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:17.272095 4314 raft_consensus.cc:515] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.272476 4314 leader_election.cc:304] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:17.274953 4314 leader_election.cc:290] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:17.275170 4316 raft_consensus.cc:2804] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:17.280138 4316 raft_consensus.cc:697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:17.280632 4316 consensus_queue.cc:237] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.284413 4314 ts_tablet_manager.cc:1434] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.022s user 0.007s sys 0.012s
I20260807 07:52:17.284814 4314 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:17.291167 4314 tablet_bootstrap.cc:654] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:17.299156 3714 catalog_manager.cc:5697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:17.299494 4314 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:17.299767 4314 ts_tablet_manager.cc:1403] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20260807 07:52:17.301117 4314 raft_consensus.cc:359] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.301497 4314 raft_consensus.cc:385] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:17.301725 4314 raft_consensus.cc:740] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:17.302166 4314 consensus_queue.cc:260] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.302433 4314 raft_consensus.cc:399] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:17.302593 4314 raft_consensus.cc:493] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:17.302724 4314 raft_consensus.cc:3060] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:17.308885 4314 raft_consensus.cc:515] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.309158 4314 leader_election.cc:304] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:17.309585 4314 leader_election.cc:290] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:17.309952 4318 raft_consensus.cc:2804] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:17.310206 4318 raft_consensus.cc:697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:17.310657 4318 consensus_queue.cc:237] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:17.313279 4231 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.14->[1.15-1.19] Dedup: 1.15->[1.16-1.19]
I20260807 07:52:17.314688 4314 ts_tablet_manager.cc:1434] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.015s user 0.000s sys 0.009s
I20260807 07:52:17.323258 3714 catalog_manager.cc:5697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:17.332393 4273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a486fca3ed6e467daa9af0e316f2e075. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:17.332685 4273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:17.332955 4310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: true } } }
I20260807 07:52:17.333177 4310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:17.337604 4310 raft_consensus.cc:1064] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: attempting to promote NON_VOTER 279e50b67caa47fb95d0c1e1beefd731 to VOTER
I20260807 07:52:17.339038 4310 consensus_queue.cc:237] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 1.22, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:17.341820 4231 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 LEARNER]: Refusing update from remote peer a486fca3ed6e467daa9af0e316f2e075: Log matching property violated. Preceding OpId in replica: term: 1 index: 22. Preceding OpId from leader: term: 1 index: 23. (index mismatch)
I20260807 07:52:17.342162 3634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Refusing update from remote peer a486fca3ed6e467daa9af0e316f2e075: Log matching property violated. Preceding OpId in replica: term: 1 index: 22. Preceding OpId from leader: term: 1 index: 23. (index mismatch)
I20260807 07:52:17.342803 4310 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260807 07:52:17.343132 3843 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260807 07:52:17.345455 3843 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 LEADER]: Committing config change with OpId 1.23: config changed from index 15 to 23, 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.346550 3634 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Committing config change with OpId 1.23: config changed from index 15 to 23, 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.347810 4231 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 FOLLOWER]: Committing config change with OpId 1.23: config changed from index 15 to 23, 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.349777 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.349946 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:17.350368 4310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.350417 3843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.350798 4310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:17.350886 3843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:17.352730 4333 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.353276 4333 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:17.354362 4273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a486fca3ed6e467daa9af0e316f2e075" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:17.354642 4273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:17.564405 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3807
I20260807 07:52:17.575943 3975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:17.675607 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3807
I20260807 07:52:17.705165 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3583
I20260807 07:52:17.706919 3659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:17.828264 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3583
I20260807 07:52:17.859651 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 3668
I20260807 07:52:17.861018 3751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:17.977252 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 3668
I20260807 07:52:18.006487 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4025
I20260807 07:52:18.007845 4262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:18.122262 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4025
I20260807 07:52:18.153004 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--webserver_interface=127.2.169.254
--webserver_port=40051
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947 with env {}
W20260807 07:52:18.521540 4345 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:18.521888 4345 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:18.522032 4345 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:18.531209 4345 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:18.531297 4345 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:18.531409 4345 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:18.531483 4345 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:18.541661 4345 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=40051
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:18.543375 4345 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:18.545456 4345 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:18.555024 4369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:18.555258 4372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:18.555676 4370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:18.556020 4345 server_base.cc:1061] running on GCE node
I20260807 07:52:18.556800 4345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:18.558270 4345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:18.559506 4345 hybrid_clock.cc:648] HybridClock initialized: now 1786089138559459 us; error 39 us; skew 500 ppm
I20260807 07:52:18.559872 4345 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:18.562320 4345 webserver.cc:533] Webserver started at http://127.2.169.254:40051/ using document root <none> and password file <none>
I20260807 07:52:18.563128 4345 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:18.563274 4345 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:18.569490 4345 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 07:52:18.572641 4379 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:18.574184 4345 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:18.574370 4345 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:18.575145 4345 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:18.600664 4345 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:18.601733 4345 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:18.602170 4345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:18.625633 4345 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:45309
I20260807 07:52:18.625634 4451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:45309 every 8 connection(s)
I20260807 07:52:18.627385 4345 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:18.635453 4452 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:18.636200 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4345
I20260807 07:52:18.637007 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--webserver_interface=127.2.169.253
--webserver_port=41101
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947 with env {}
I20260807 07:52:18.640122 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap starting.
I20260807 07:52:18.667308 4452 log.cc:826] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:18.696233 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:18.696950 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap complete.
I20260807 07:52:18.705950 4452 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:18.706914 4452 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Initialized, Role: FOLLOWER
I20260807 07:52:18.707759 4452 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:18.708755 4460 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:18.709088 4460 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:18.710093 4452 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:18.729506 4477 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:18.729707 4477 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: loading cluster ID for follower catalog manager: success
I20260807 07:52:18.733779 4477 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: acquiring CA information for follower catalog manager: success
I20260807 07:52:18.736564 4477 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:18.731144 4345 master_runner.cc:187] Error getting master registration for 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111),
I20260807 07:52:18.737223 4345 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:19.037508 4455 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:19.038028 4455 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:19.038213 4455 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:19.047667 4455 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:19.047763 4455 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:19.047868 4455 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:19.047931 4455 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:19.059145 4455 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=41101
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:19.061065 4455 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:19.063268 4455 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:19.072938 4488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:19.072942 4486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:19.078076 4490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:19.078322 4455 server_base.cc:1061] running on GCE node
I20260807 07:52:19.079411 4455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:19.081188 4455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:19.082437 4455 hybrid_clock.cc:648] HybridClock initialized: now 1786089139082359 us; error 61 us; skew 500 ppm
I20260807 07:52:19.082840 4455 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:19.085752 4455 webserver.cc:533] Webserver started at http://127.2.169.253:41101/ using document root <none> and password file <none>
I20260807 07:52:19.087069 4455 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:19.087272 4455 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:19.095305 4455 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 07:52:19.098711 4498 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:19.100481 4455 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 07:52:19.100682 4455 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:19.101667 4455 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:19.128314 4455 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:19.129418 4455 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:19.129777 4455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:19.155151 4455 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:45589
I20260807 07:52:19.155170 4562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:45589 every 8 connection(s)
I20260807 07:52:19.157130 4455 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:19.162315 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4455
I20260807 07:52:19.163110 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--webserver_interface=127.2.169.252
--webserver_port=46357
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947 with env {}
I20260807 07:52:19.166132 4565 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:19.171238 4565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap starting.
I20260807 07:52:19.196573 4565 log.cc:826] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:19.222265 4565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:19.222924 4565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap complete.
I20260807 07:52:19.229210 4565 raft_consensus.cc:359] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.229979 4565 raft_consensus.cc:740] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a486fca3ed6e467daa9af0e316f2e075, State: Initialized, Role: FOLLOWER
I20260807 07:52:19.230628 4565 consensus_queue.cc:260] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.231549 4574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:19.231882 4574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:19.232564 4565 sys_catalog.cc:565] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:19.257040 4591 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:19.257231 4591 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: loading cluster ID for follower catalog manager: success
I20260807 07:52:19.262981 4591 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: acquiring CA information for follower catalog manager: success
I20260807 07:52:19.266103 4591 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:19.274983 4455 master_runner.cc:187] Error getting master registration for 127.2.169.252:39947: Network error: Client connection negotiation failed: client connection to 127.2.169.252:39947: connect: Connection refused (error 111),
I20260807 07:52:19.280833 4455 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:19.572438 4567 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:19.572795 4567 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:19.572929 4567 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:19.582285 4567 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:19.582413 4567 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:19.582523 4567 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:19.582599 4567 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:19.592779 4567 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=46357
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:19.594620 4567 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:19.596568 4567 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:19.605180 4607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:19.605255 4606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:19.606237 4610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:19.607878 4567 server_base.cc:1061] running on GCE node
I20260807 07:52:19.608796 4567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:19.610381 4567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:19.611629 4567 hybrid_clock.cc:648] HybridClock initialized: now 1786089139611519 us; error 101 us; skew 500 ppm
I20260807 07:52:19.612008 4567 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:19.614500 4567 webserver.cc:533] Webserver started at http://127.2.169.252:46357/ using document root <none> and password file <none>
I20260807 07:52:19.615267 4567 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:19.615409 4567 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:19.621508 4567 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 07:52:19.625572 4619 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:19.627425 4567 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 07:52:19.627604 4567 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:19.628592 4567 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:19.660547 4567 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:19.661878 4567 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:19.662267 4567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:19.688421 4567 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:39947
I20260807 07:52:19.688448 4683 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:39947 every 8 connection(s)
I20260807 07:52:19.690294 4567 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:19.698750 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4567
I20260807 07:52:19.699137 4684 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:19.699735 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:36591
--local_ip_for_outbound_sockets=127.2.169.193
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589
--webserver_port=34149
--webserver_interface=127.2.169.193
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947 with env {}
I20260807 07:52:19.703331 4684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap starting.
I20260807 07:52:19.730719 4684 log.cc:826] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:19.775229 4684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:19.775877 4684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap complete.
I20260807 07:52:19.782423 4684 raft_consensus.cc:359] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.783116 4684 raft_consensus.cc:740] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e50b67caa47fb95d0c1e1beefd731, State: Initialized, Role: FOLLOWER
I20260807 07:52:19.783833 4684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.784861 4691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:19.785219 4691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:19.785966 4684 sys_catalog.cc:565] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:19.808705 4704 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:19.808827 4704 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: loading cluster ID for follower catalog manager: success
I20260807 07:52:19.817411 4704 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: acquiring CA information for follower catalog manager: success
I20260807 07:52:19.820186 4704 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:19.845027 4567 master_runner.cc:264] No leader master found from master 279e50b67caa47fb95d0c1e1beefd731
I20260807 07:52:19.845182 4567 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260807 07:52:19.875715 4716 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:19.876076 4716 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.878399 4716 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947)
I20260807 07:52:19.888548 4652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731" is_pre_election: true
I20260807 07:52:19.889168 4652 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 1.
I20260807 07:52:19.889611 4528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075" is_pre_election: true
I20260807 07:52:19.889961 4384 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 279e50b67caa47fb95d0c1e1beefd731, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:19.890138 4528 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 1.
I20260807 07:52:19.890404 4716 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 07:52:19.890568 4716 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:19.890686 4716 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:19.894454 4716 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.895570 4528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075"
I20260807 07:52:19.895645 4652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 2 candidate_status { last_received { term: 1 index: 23 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731"
I20260807 07:52:19.895776 4528 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:19.895830 4652 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:19.901247 4528 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 2.
I20260807 07:52:19.901252 4652 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 2.
I20260807 07:52:19.901831 4382 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:19.902053 4716 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 2 election: Requested vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947)
I20260807 07:52:19.902297 4716 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:19.902477 4716 raft_consensus.cc:697] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 LEADER]: Becoming Leader. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Running, Role: LEADER
I20260807 07:52:19.903111 4716 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 1.23, Last appended by leader: 23, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } }
I20260807 07:52:19.911088 4716 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:19.911376 4716 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:19.913410 4723 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:19.916961 4723 catalog_manager.cc:679] Loaded metadata for table default.1_2 [id=44b956d4571840a780d714244fff640c]
I20260807 07:52:19.917512 4723 catalog_manager.cc:679] Loaded metadata for table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84]
I20260807 07:52:19.917888 4723 catalog_manager.cc:679] Loaded metadata for table test-workload [id=e956c0fa574044c6b86ec65f8705a989]
I20260807 07:52:19.918248 4723 catalog_manager.cc:679] Loaded metadata for table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c]
I20260807 07:52:19.918592 4723 catalog_manager.cc:679] Loaded metadata for table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e]
I20260807 07:52:19.920469 4723 tablet_loader.cc:96] loaded metadata for tablet 23de065eeead41b8aaec4866731369f5 (table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84])
I20260807 07:52:19.920991 4723 tablet_loader.cc:96] loaded metadata for tablet 7c30a1b53ff54e9a8515149bd3d5d84f (table test-workload [id=e956c0fa574044c6b86ec65f8705a989])
I20260807 07:52:19.923418 4723 tablet_loader.cc:96] loaded metadata for tablet ba7a33d575f4453dbf1b8861634a3f98 (table default.1_2 [id=44b956d4571840a780d714244fff640c])
I20260807 07:52:19.924095 4723 tablet_loader.cc:96] loaded metadata for tablet de31825f9a984012abe2d537ea2b84f5 (table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e])
I20260807 07:52:19.924714 4723 tablet_loader.cc:96] loaded metadata for tablet fda7602334414a979800d942a24cfb86 (table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c])
I20260807 07:52:19.925818 4723 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:19.926788 4723 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:19.926895 4723 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:19.928087 4723 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:19.928934 4723 catalog_manager.cc:6081] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Loaded TSK: 0
I20260807 07:52:19.929839 4723 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:19.988343 4401 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32832:
name: "default.0_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:19.988358 4399 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32820:
name: "default.1_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 07:52:20.170574 4686 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:20.170920 4686 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:20.171085 4686 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:20.180104 4686 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:20.180336 4686 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:20.191378 4686 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:36591
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=34149
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:20.193064 4686 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:20.195039 4686 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:20.205130 4728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:20.205152 4730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:20.206632 4734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:20.207676 4686 server_base.cc:1061] running on GCE node
I20260807 07:52:20.208323 4686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:20.209595 4686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:20.210825 4686 hybrid_clock.cc:648] HybridClock initialized: now 1786089140210753 us; error 60 us; skew 500 ppm
I20260807 07:52:20.211278 4686 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:20.213860 4686 webserver.cc:533] Webserver started at http://127.2.169.193:34149/ using document root <none> and password file <none>
I20260807 07:52:20.214732 4686 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:20.214890 4686 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:20.220707 4686 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 07:52:20.224249 4744 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:20.225834 4686 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 07:52:20.226025 4686 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:20.226743 4686 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:20.247817 4686 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:20.248803 4686 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:20.249128 4686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:20.250300 4686 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:20.252910 4754 ts_tablet_manager.cc:548] Loading tablet metadata (0/5 complete)
I20260807 07:52:20.272747 4686 ts_tablet_manager.cc:585] Loaded tablet metadata (5 total tablets, 5 live tablets)
I20260807 07:52:20.272850 4686 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.021s user 0.001s sys 0.001s
I20260807 07:52:20.273001 4686 ts_tablet_manager.cc:600] Registering tablets (0/5 complete)
I20260807 07:52:20.276899 4754 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:20.287140 4686 ts_tablet_manager.cc:616] Registered 5 tablets
I20260807 07:52:20.287299 4686 ts_tablet_manager.cc:595] Time spent register tablets: real 0.014s user 0.010s sys 0.003s
I20260807 07:52:20.302695 4754 log.cc:826] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:20.308741 4754 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:20.309751 4754 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:20.311980 4754 ts_tablet_manager.cc:1403] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.036s user 0.017s sys 0.015s
I20260807 07:52:20.324915 4754 raft_consensus.cc:359] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.326295 4754 raft_consensus.cc:740] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:20.327775 4754 consensus_queue.cc:260] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.328094 4754 raft_consensus.cc:399] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:20.328302 4754 raft_consensus.cc:493] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:20.328529 4754 raft_consensus.cc:3060] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:20.336659 4754 raft_consensus.cc:515] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.337512 4754 leader_election.cc:304] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:20.340571 4754 leader_election.cc:290] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:20.340708 4801 raft_consensus.cc:2804] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:20.344065 4754 ts_tablet_manager.cc:1434] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.032s user 0.026s sys 0.005s
I20260807 07:52:20.344434 4801 raft_consensus.cc:697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:20.344673 4754 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:20.345170 4801 consensus_queue.cc:237] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.348878 4528 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 2 index: 24. (index mismatch)
I20260807 07:52:20.350137 4716 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260807 07:52:20.366721 4754 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:20.367344 4754 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:20.367686 4815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.367980 4815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:20.368848 4754 ts_tablet_manager.cc:1403] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.003s
I20260807 07:52:20.370599 4716 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.370904 4716 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:20.372016 4652 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 1 index: 23. Preceding OpId from leader: term: 2 index: 24. (index mismatch)
I20260807 07:52:20.373257 4716 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260807 07:52:20.376199 4754 raft_consensus.cc:359] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.376385 4754 raft_consensus.cc:740] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:20.376667 4754 consensus_queue.cc:260] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.376883 4754 raft_consensus.cc:399] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:20.376986 4754 raft_consensus.cc:493] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:20.377100 4754 raft_consensus.cc:3060] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:20.379711 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.380069 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:20.380781 4754 raft_consensus.cc:515] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.380743 4844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.381073 4754 leader_election.cc:304] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:20.381475 4809 raft_consensus.cc:2804] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:20.381641 4809 raft_consensus.cc:697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:20.382041 4809 consensus_queue.cc:237] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.382599 4844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:20.383204 4754 leader_election.cc:290] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:20.383901 4754 ts_tablet_manager.cc:1434] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.015s user 0.002s sys 0.004s
I20260807 07:52:20.384259 4754 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:20.393356 4815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.393656 4815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:20.407366 4754 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:20.408289 4754 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:20.409855 4754 ts_tablet_manager.cc:1403] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.014s
I20260807 07:52:20.411048 4754 raft_consensus.cc:359] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.411298 4754 raft_consensus.cc:740] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:20.411844 4754 consensus_queue.cc:260] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.412081 4754 raft_consensus.cc:399] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:20.412199 4754 raft_consensus.cc:493] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:20.412325 4754 raft_consensus.cc:3060] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:20.414477 4686 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:36591
I20260807 07:52:20.414569 4917 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:36591 every 8 connection(s)
I20260807 07:52:20.415445 4754 raft_consensus.cc:515] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.415678 4754 leader_election.cc:304] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:20.415939 4754 leader_election.cc:290] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:20.415993 4809 raft_consensus.cc:2804] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:20.416188 4809 raft_consensus.cc:697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:20.416262 4686 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:20.416500 4754 ts_tablet_manager.cc:1434] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260807 07:52:20.416463 4809 consensus_queue.cc:237] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.416868 4754 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:20.423906 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4686
I20260807 07:52:20.453013 4918 heartbeater.cc:344] Connected to a master server at 127.2.169.252:39947
I20260807 07:52:20.453943 4918 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:20.456007 4918 heartbeater.cc:507] Master 127.2.169.252:39947 requested a full tablet report, sending...
I20260807 07:52:20.464138 4639 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:20.469508 4921 heartbeater.cc:344] Connected to a master server at 127.2.169.254:45309
I20260807 07:52:20.469713 4921 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:20.469837 4754 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:20.470495 4921 heartbeater.cc:507] Master 127.2.169.254:45309 requested a full tablet report, sending...
I20260807 07:52:20.470539 4754 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:20.471928 4754 ts_tablet_manager.cc:1403] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.055s user 0.015s sys 0.028s
I20260807 07:52:20.472877 4754 raft_consensus.cc:359] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.473054 4754 raft_consensus.cc:740] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:20.473322 4754 consensus_queue.cc:260] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.473557 4754 raft_consensus.cc:399] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:20.473682 4754 raft_consensus.cc:493] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:20.473798 4754 raft_consensus.cc:3060] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:20.474025 4399 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:20.477411 4754 raft_consensus.cc:515] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.477913 4754 leader_election.cc:304] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:20.478188 4754 leader_election.cc:290] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:20.478241 4809 raft_consensus.cc:2804] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:20.478375 4809 raft_consensus.cc:697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:20.478614 4754 ts_tablet_manager.cc:1434] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 07:52:20.478643 4809 consensus_queue.cc:237] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.478927 4754 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:20.480911 4399 catalog_manager.cc:5697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:20.482064 4399 catalog_manager.cc:5697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:20.485266 4399 catalog_manager.cc:5697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:20.498960 4720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } }
I20260807 07:52:20.499234 4720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:20.503891 4922 heartbeater.cc:344] Connected to a master server at 127.2.169.253:45589
I20260807 07:52:20.504091 4922 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:20.504745 4922 heartbeater.cc:507] Master 127.2.169.253:45589 requested a full tablet report, sending...
I20260807 07:52:20.527244 4754 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:20.527211 4516 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:20.527810 4754 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:20.528885 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:20.529207 4754 ts_tablet_manager.cc:1403] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.050s user 0.030s sys 0.016s
I20260807 07:52:20.529955 4754 raft_consensus.cc:359] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.530143 4754 raft_consensus.cc:740] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:20.530401 4754 consensus_queue.cc:260] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.530614 4754 raft_consensus.cc:399] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:20.530730 4754 raft_consensus.cc:493] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:20.530833 4754 raft_consensus.cc:3060] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:20.534098 4754 raft_consensus.cc:515] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.534332 4754 leader_election.cc:304] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:20.534582 4754 leader_election.cc:290] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:20.534619 4809 raft_consensus.cc:2804] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:20.534833 4809 raft_consensus.cc:697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:20.535085 4754 ts_tablet_manager.cc:1434] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260807 07:52:20.535094 4809 consensus_queue.cc:237] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:20.539245 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:20.540388 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--webserver_interface=127.2.169.251
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
I20260807 07:52:20.543164 4399 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:36451
I20260807 07:52:20.550907 4921 heartbeater.cc:499] Master 127.2.169.254:45309 was elected leader, sending a full tablet report...
I20260807 07:52:20.554060 4399 catalog_manager.cc:5697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:20.554556 4399 catalog_manager.cc:5697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:21.086314 4399 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32878:
name: "default.1_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:21.113308 4401 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32880:
name: "default.0_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:21.178575 4807 tablet_service.cc:1511] Processing CreateTablet for tablet dea58cd0bc524d3e842a2feb335a6bfe (DEFAULT_TABLE table=default.1_4 [id=d44dab2abaf647b0a118ad2af02204d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:21.179864 4807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea58cd0bc524d3e842a2feb335a6bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:21.191284 4806 tablet_service.cc:1511] Processing CreateTablet for tablet bcabb79073b040bfa91e1733e3d96128 (DEFAULT_TABLE table=default.0_4 [id=b198b5c57b4d4f89a279ae74df9fe9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:21.191829 4806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcabb79073b040bfa91e1733e3d96128. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 07:52:21.198309 4948 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:21.198765 4948 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:21.198902 4948 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260807 07:52:21.201519 4993 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:21.203326 4993 tablet_bootstrap.cc:654] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:21.206714 4993 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:21.206908 4993 ts_tablet_manager.cc:1403] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:52:21.207965 4993 raft_consensus.cc:359] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.208133 4993 raft_consensus.cc:385] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:21.208254 4993 raft_consensus.cc:740] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:21.208614 4993 consensus_queue.cc:260] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.208818 4993 raft_consensus.cc:399] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:21.208913 4993 raft_consensus.cc:493] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:21.209033 4993 raft_consensus.cc:3060] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
W20260807 07:52:21.210441 4948 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:21.210570 4948 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:21.210662 4948 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:21.210747 4948 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:21.211730 4993 raft_consensus.cc:515] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.212013 4993 leader_election.cc:304] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:21.212821 4993 leader_election.cc:290] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:21.213016 4995 raft_consensus.cc:2804] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:21.213637 4995 raft_consensus.cc:697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:21.214074 4995 consensus_queue.cc:237] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.219931 4993 ts_tablet_manager.cc:1434] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 07:52:21.220256 4993 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:21.222239 4399 catalog_manager.cc:5697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:21.222811 4993 tablet_bootstrap.cc:654] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:21.226331 4948 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.2.169.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:21.227406 4993 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:21.227667 4993 ts_tablet_manager.cc:1403] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 07:52:21.228544 4993 raft_consensus.cc:359] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.228782 4948 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:21.228796 4993 raft_consensus.cc:385] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:21.229044 4993 raft_consensus.cc:740] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:21.229558 4993 consensus_queue.cc:260] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.229799 4993 raft_consensus.cc:399] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:21.229920 4993 raft_consensus.cc:493] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:21.230054 4993 raft_consensus.cc:3060] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:21.231554 4948 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260807 07:52:21.234998 4993 raft_consensus.cc:515] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.235249 4993 leader_election.cc:304] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:21.235519 4995 raft_consensus.cc:2804] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:21.235705 4995 raft_consensus.cc:697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:21.235985 4995 consensus_queue.cc:237] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:21.236589 4993 leader_election.cc:290] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
W20260807 07:52:21.237193 4933 tablet.cc:2406] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:21.245091 4993 ts_tablet_manager.cc:1434] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
W20260807 07:52:21.250952 5008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:21.255483 5009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:21.261251 4401 catalog_manager.cc:5697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:21.267513 4948 server_base.cc:1061] running on GCE node
W20260807 07:52:21.268086 5011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:21.269443 4948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:21.271564 4948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:21.272857 4948 hybrid_clock.cc:648] HybridClock initialized: now 1786089141272773 us; error 75 us; skew 500 ppm
I20260807 07:52:21.273324 4948 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:21.276271 4948 webserver.cc:533] Webserver started at http://127.2.169.251:43555/ using document root <none> and password file <none>
I20260807 07:52:21.277148 4948 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:21.277375 4948 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:21.277974 4948 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:21.280803 4948 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/instance:
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:21.281816 4948 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal/instance:
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:21.288164 4948 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 07:52:21.292024 5018 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:21.293552 4948 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:21.293763 4948 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:21.293993 4948 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:21.322072 4948 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:21.323197 4948 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:21.323550 4948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:21.347949 5095 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.251:34401 every 8 connection(s)
I20260807 07:52:21.347958 4948 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.251:34401
I20260807 07:52:21.349864 4948 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
I20260807 07:52:21.354444 5096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:21.356273 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 4948
I20260807 07:52:21.361650 5096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:21.376959 5096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:21.383718 5096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:21.390985 5096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:21.404827 5096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap starting.
I20260807 07:52:21.408645 5096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:21.409816 5096 log.cc:826] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:21.412837 5096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: No bootstrap required, opened a new log
I20260807 07:52:21.420886 5096 raft_consensus.cc:359] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } }
I20260807 07:52:21.421319 5096 raft_consensus.cc:385] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:21.421479 5096 raft_consensus.cc:740] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea850e24aa3845a08ce29d1510612bb7, State: Initialized, Role: FOLLOWER
I20260807 07:52:21.422273 5096 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } }
I20260807 07:52:21.423463 5101 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } } }
I20260807 07:52:21.423879 5101 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:21.424712 5096 sys_catalog.cc:565] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:21.437033 5113 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:21.437181 5113 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:21.443218 4948 master_runner.cc:429] Detected that this master ea850e24aa3845a08ce29d1510612bb7 is joining an existing cluster
I20260807 07:52:21.443351 4948 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260807 07:52:21.444295 4948 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:52:21.444655 4948 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 07:52:21.444880 4948 tablet_replica.cc:333] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: stopping tablet replica
I20260807 07:52:21.446544 4948 master_runner.cc:434] Initiating AddMaster RPC to add 127.2.169.251:34401
I20260807 07:52:21.501157 4401 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.2.169.251:34401: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7"
member_type: NON_VOTER
last_known_addr {
host: "127.2.169.251"
port: 34401
}
attrs {
promote: true
}
}
dest_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
cas_config_opid_index: 23
I20260807 07:52:21.502712 4401 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 23, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } }
I20260807 07:52:21.505184 4652 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20260807 07:52:21.505201 4527 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20260807 07:52:21.505959 4976 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
W20260807 07:52:21.506273 4384 consensus_peers.cc:597] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a -> Peer ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401): Couldn't send request to peer ea850e24aa3845a08ce29d1510612bb7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:21.506659 4976 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20260807 07:52:21.509039 4986 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 23 to 34, NON_VOTER ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.511194 4652 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 23 to 34, NON_VOTER ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.512789 4986 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.2.169.251:34401
I20260807 07:52:21.513196 4941 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.513674 4948 master.cc:562] Master@127.2.169.251:34401 shutting down...
I20260807 07:52:21.513365 4986 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.510429 4527 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 23 to 34, NON_VOTER ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.513538 4941 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:21.517406 4986 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:21.519625 4941 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.519878 4941 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:21.528396 5126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.528837 5126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:21.533179 4948 master.cc:584] Master@127.2.169.251:34401 shutdown complete.
I20260807 07:52:21.533304 4948 master_runner.cc:306] Clearing existing system tablet
I20260807 07:52:21.533823 4948 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:21.534070 4948 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:21.534258 5127 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.534700 5127 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 07:52:21.537743 4401 master.cc:680] Network error: unable to get registration information for peer ea850e24aa3845a08ce29d1510612bb7: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111)
I20260807 07:52:21.538273 4948 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 07:52:21.540045 5129 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:21.540444 4948 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:21.540746 4948 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 07:52:21.540879 4948 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:21.544178 4948 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 07:52:21.549094 4948 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 07:52:21.549273 4948 log.cc:1199] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Deleting WAL directory at /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260807 07:52:21.549983 4948 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Deleting consensus metadata
I20260807 07:52:21.550283 4948 master_runner.cc:316] Copying system tablet from 127.2.169.254:45309
I20260807 07:52:21.555056 4948 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: tablet copy: Beginning tablet copy session from remote peer at address 127.2.169.254:45309
I20260807 07:52:21.567066 4430 tablet_copy_service.cc:140] P ca5ccffd14db4e15be5ec8b65fc2502a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ea850e24aa3845a08ce29d1510612bb7 ({username='slave'} at 127.0.0.1:32910)
I20260807 07:52:21.567708 4430 tablet_copy_service.cc:161] P ca5ccffd14db4e15be5ec8b65fc2502a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ea850e24aa3845a08ce29d1510612bb7 at {username='slave'} at 127.0.0.1:32910: session id = ea850e24aa3845a08ce29d1510612bb7-00000000000000000000000000000000
I20260807 07:52:21.571266 4430 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 07:52:21.573683 4948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 07:52:21.579620 4948 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: tablet copy: Starting download of 0 data blocks...
I20260807 07:52:21.579906 4948 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: tablet copy: Starting download of 1 WAL segments...
I20260807 07:52:21.583809 4948 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 07:52:21.587410 4430 tablet_copy_service.cc:342] P ca5ccffd14db4e15be5ec8b65fc2502a: Request end of tablet copy session ea850e24aa3845a08ce29d1510612bb7-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:32910
I20260807 07:52:21.587769 4430 tablet_copy_service.cc:434] P ca5ccffd14db4e15be5ec8b65fc2502a: ending tablet copy session ea850e24aa3845a08ce29d1510612bb7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ea850e24aa3845a08ce29d1510612bb7
W20260807 07:52:21.595084 4948 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260807 07:52:21.598680 4948 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:21.601276 5137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:21.602056 5138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:21.607184 4948 server_base.cc:1061] running on GCE node
W20260807 07:52:21.607515 5140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:21.608026 4948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:21.608671 4948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:21.609879 4948 hybrid_clock.cc:648] HybridClock initialized: now 1786089141609802 us; error 43 us; skew 500 ppm
I20260807 07:52:21.610138 4948 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:21.611917 4948 webserver.cc:533] Webserver started at http://127.2.169.251:45695/ using document root <none> and password file <none>
I20260807 07:52:21.612303 4948 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:21.612447 4948 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:21.615159 4948 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 07:52:21.617322 5149 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:21.617834 4948 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 07:52:21.617957 4948 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:21.618155 4948 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:21.641847 4948 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:21.643399 4948 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260807 07:52:21.674906 4401 master.cc:680] Network error: unable to get registration information for peer ea850e24aa3845a08ce29d1510612bb7: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111)
I20260807 07:52:21.759886 4948 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.251:34401
I20260807 07:52:21.760326 5223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.251:34401 every 8 connection(s)
I20260807 07:52:21.761714 4948 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
I20260807 07:52:21.769009 5224 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:21.772418 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap starting.
I20260807 07:52:21.817510 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=17 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:21.818295 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap complete.
I20260807 07:52:21.820072 5224 raft_consensus.cc:359] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } }
I20260807 07:52:21.820410 5224 raft_consensus.cc:740] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: ea850e24aa3845a08ce29d1510612bb7, State: Initialized, Role: LEARNER
I20260807 07:52:21.820818 5224 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } }
I20260807 07:52:21.822886 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.823411 5224 sys_catalog.cc:565] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:21.823618 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:21.849583 5243 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:21.849814 5243 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: loading cluster ID for follower catalog manager: success
I20260807 07:52:21.853662 5243 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: acquiring CA information for follower catalog manager: success
I20260807 07:52:21.856537 5243 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:21.923725 5186 raft_consensus.cc:1217] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.33->[2.34-2.34] Dedup: 2.34->[]
I20260807 07:52:21.924856 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:21.925670 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:22.303797 4401 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32922:
name: "default.1_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:22.332065 5269 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: NON_VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: true } } }
I20260807 07:52:22.332639 5269 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:22.344025 5267 raft_consensus.cc:1064] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: attempting to promote NON_VOTER ea850e24aa3845a08ce29d1510612bb7 to VOTER
I20260807 07:52:22.345108 5267 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 23, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:22.348763 4401 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52798:
name: "default.0_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:22.350899 4527 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260807 07:52:22.350939 5186 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 LEARNER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260807 07:52:22.351179 4652 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20260807 07:52:22.351678 5279 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.001s
I20260807 07:52:22.352596 5269 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20260807 07:52:22.352881 5270 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20260807 07:52:22.357458 5279 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 34 to 36, ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.358377 5186 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 34 to 36, ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.360740 4527 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 34 to 36, ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.364200 4807 tablet_service.cc:1511] Processing CreateTablet for tablet 1298cbdc19e640b4bbbeeaf1c34d73bf (DEFAULT_TABLE table=default.1_5 [id=f2e18629f8db4f5f9e360d55d59d8526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:22.364979 4807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1298cbdc19e640b4bbbeeaf1c34d73bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:22.368404 5267 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.368294 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.368733 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:22.368821 5267 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:22.369107 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.369352 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:22.369462 5270 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.369803 5270 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:22.369257 4652 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 34 to 36, ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.387609 5290 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:22.389528 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.389882 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:22.391966 5287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.392417 5287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:22.394790 5288 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:22.395370 5288 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:22.396814 5290 tablet_bootstrap.cc:654] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:22.400501 4807 tablet_service.cc:1511] Processing CreateTablet for tablet e25a88e31cf744e8b38bc2996babb16a (DEFAULT_TABLE table=default.0_5 [id=73efb1d25ad24e9bb7ad4fcd27db89e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:22.401033 4807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e25a88e31cf744e8b38bc2996babb16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:22.402195 5290 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:22.402454 5290 ts_tablet_manager.cc:1403] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.004s
I20260807 07:52:22.404016 5290 raft_consensus.cc:359] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.404269 5290 raft_consensus.cc:385] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:22.404484 5290 raft_consensus.cc:740] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:22.404954 5290 consensus_queue.cc:260] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.405244 5290 raft_consensus.cc:399] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:22.405412 5290 raft_consensus.cc:493] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:22.405591 5290 raft_consensus.cc:3060] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:22.408761 5290 raft_consensus.cc:515] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.409076 5290 leader_election.cc:304] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:22.413859 5290 leader_election.cc:290] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:22.414153 5295 raft_consensus.cc:2804] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:22.415166 5295 raft_consensus.cc:697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:22.415663 5295 consensus_queue.cc:237] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.420903 4399 catalog_manager.cc:5697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:22.431198 5290 ts_tablet_manager.cc:1434] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.028s user 0.000s sys 0.025s
I20260807 07:52:22.431680 5290 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:22.436197 5290 tablet_bootstrap.cc:654] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:22.445992 5290 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:22.446264 5290 ts_tablet_manager.cc:1403] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.002s
I20260807 07:52:22.447564 5290 raft_consensus.cc:359] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.447855 5290 raft_consensus.cc:385] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:22.447990 5290 raft_consensus.cc:740] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:22.448417 5290 consensus_queue.cc:260] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.448714 5290 raft_consensus.cc:399] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:22.448904 5290 raft_consensus.cc:493] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:22.449102 5290 raft_consensus.cc:3060] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:22.453663 5290 raft_consensus.cc:515] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.454015 5290 leader_election.cc:304] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:22.454456 5290 leader_election.cc:290] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:22.455116 5290 ts_tablet_manager.cc:1434] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260807 07:52:22.455498 5295 raft_consensus.cc:2804] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:22.455914 5295 raft_consensus.cc:697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:22.456284 5295 consensus_queue.cc:237] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:22.462730 4401 catalog_manager.cc:5697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:22.563563 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4686
I20260807 07:52:22.575980 4912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:22.681993 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4686
I20260807 07:52:22.712322 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4345
I20260807 07:52:22.713896 4444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:22.834576 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4345
I20260807 07:52:22.867977 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4455
I20260807 07:52:22.869279 4556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:22.984300 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4455
I20260807 07:52:23.016695 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4567
I20260807 07:52:23.018035 4678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:23.132990 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4567
I20260807 07:52:23.165617 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 4948
I20260807 07:52:23.166819 5213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:23.281433 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 4948
I20260807 07:52:23.312707 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--webserver_interface=127.2.169.254
--webserver_port=40051
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
W20260807 07:52:23.694370 5316 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:23.694854 5316 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:23.694994 5316 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:23.708796 5316 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:23.708948 5316 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:23.709069 5316 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:23.709149 5316 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:23.726011 5316 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=40051
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:23.728295 5316 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:23.731259 5316 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:23.741911 5342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:23.742350 5344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:23.743052 5341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:23.746336 5316 server_base.cc:1061] running on GCE node
I20260807 07:52:23.747145 5316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:23.748605 5316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:23.749837 5316 hybrid_clock.cc:648] HybridClock initialized: now 1786089143749790 us; error 57 us; skew 500 ppm
I20260807 07:52:23.750320 5316 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:23.753437 5316 webserver.cc:533] Webserver started at http://127.2.169.254:40051/ using document root <none> and password file <none>
I20260807 07:52:23.754354 5316 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:23.754519 5316 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:23.761111 5316 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 07:52:23.763990 5350 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:23.765524 5316 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:23.765730 5316 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:23.766534 5316 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:23.789701 5316 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:23.790613 5316 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:23.790937 5316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:23.814204 5316 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:45309
I20260807 07:52:23.814224 5414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:45309 every 8 connection(s)
I20260807 07:52:23.815968 5316 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:23.817171 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 5316
I20260807 07:52:23.817884 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--webserver_interface=127.2.169.253
--webserver_port=41101
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
I20260807 07:52:23.824647 5415 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:23.829967 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap starting.
I20260807 07:52:23.860399 5415 log.cc:826] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:23.895503 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:23.896138 5415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap complete.
I20260807 07:52:23.902621 5415 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:23.903352 5415 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Initialized, Role: FOLLOWER
I20260807 07:52:23.904021 5415 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:23.905061 5424 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:23.905623 5424 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:23.906342 5415 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:23.918963 5436 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:23.919152 5436 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: loading cluster ID for follower catalog manager: success
I20260807 07:52:23.923660 5436 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: acquiring CA information for follower catalog manager: success
I20260807 07:52:23.926499 5436 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:23.938827 5316 master_runner.cc:187] Error getting master registration for 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111),
I20260807 07:52:23.944595 5316 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:24.210615 5417 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:24.210923 5417 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:24.211077 5417 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:24.221530 5417 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:24.221629 5417 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:24.221717 5417 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:24.221784 5417 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:24.232340 5417 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=41101
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:24.234192 5417 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:24.236112 5417 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:24.244515 5445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:24.244554 5447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:24.245649 5449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:24.246940 5417 server_base.cc:1061] running on GCE node
I20260807 07:52:24.247964 5417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:24.249941 5417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:24.251207 5417 hybrid_clock.cc:648] HybridClock initialized: now 1786089144251151 us; error 59 us; skew 500 ppm
I20260807 07:52:24.251662 5417 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:24.255705 5417 webserver.cc:533] Webserver started at http://127.2.169.253:41101/ using document root <none> and password file <none>
I20260807 07:52:24.256600 5417 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:24.256762 5417 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:24.262843 5417 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 07:52:24.266179 5455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:24.267699 5417 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:24.267904 5417 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:24.268714 5417 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:24.290246 5417 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:24.291157 5417 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:24.291515 5417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:24.315934 5417 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:45589
I20260807 07:52:24.315933 5522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:45589 every 8 connection(s)
I20260807 07:52:24.317965 5417 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:24.319630 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 5417
I20260807 07:52:24.320405 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--webserver_interface=127.2.169.252
--webserver_port=46357
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
I20260807 07:52:24.328205 5523 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:24.332078 5523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap starting.
I20260807 07:52:24.366171 5523 log.cc:826] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:24.401297 5523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:24.401955 5523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap complete.
I20260807 07:52:24.408243 5523 raft_consensus.cc:359] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:24.408959 5523 raft_consensus.cc:740] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a486fca3ed6e467daa9af0e316f2e075, State: Initialized, Role: FOLLOWER
I20260807 07:52:24.409705 5523 consensus_queue.cc:260] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:24.410687 5530 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:24.411125 5530 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:24.411792 5523 sys_catalog.cc:565] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:24.424961 5548 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:24.425159 5548 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: loading cluster ID for follower catalog manager: success
I20260807 07:52:24.430470 5548 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: acquiring CA information for follower catalog manager: success
I20260807 07:52:24.433557 5548 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:24.448101 5417 master_runner.cc:187] Error getting master registration for 127.2.169.252:39947: Network error: Client connection negotiation failed: client connection to 127.2.169.252:39947: connect: Connection refused (error 111),
I20260807 07:52:24.453892 5417 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:24.712292 5525 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:24.712608 5525 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:24.712783 5525 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:24.721570 5525 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:24.721653 5525 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:24.721738 5525 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:24.721825 5525 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:24.733830 5525 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=46357
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:24.735528 5525 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:24.737512 5525 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:24.745800 5557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:24.746495 5558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:24.747364 5561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:24.748507 5525 server_base.cc:1061] running on GCE node
I20260807 07:52:24.749307 5525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:24.750840 5525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:24.752067 5525 hybrid_clock.cc:648] HybridClock initialized: now 1786089144752030 us; error 28 us; skew 500 ppm
I20260807 07:52:24.752427 5525 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:24.754992 5525 webserver.cc:533] Webserver started at http://127.2.169.252:46357/ using document root <none> and password file <none>
I20260807 07:52:24.755759 5525 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:24.755859 5525 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:24.761950 5525 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 07:52:24.765297 5569 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:24.766878 5525 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:24.767104 5525 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:24.767907 5525 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:24.792483 5525 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:24.793604 5525 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:24.793932 5525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:24.820708 5525 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:39947
I20260807 07:52:24.820793 5634 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:39947 every 8 connection(s)
I20260807 07:52:24.823065 5525 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:24.833438 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 5525
I20260807 07:52:24.834182 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--webserver_interface=127.2.169.251
--webserver_port=43555
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
I20260807 07:52:24.839087 5635 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:24.844470 5635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap starting.
I20260807 07:52:24.872184 5635 log.cc:826] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:24.904965 5635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:24.905622 5635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap complete.
I20260807 07:52:24.911980 5635 raft_consensus.cc:359] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:24.912940 5635 raft_consensus.cc:740] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e50b67caa47fb95d0c1e1beefd731, State: Initialized, Role: FOLLOWER
I20260807 07:52:24.913800 5635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:24.917747 5643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:24.918109 5643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:24.918817 5635 sys_catalog.cc:565] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:24.932889 5657 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:24.933065 5657 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: loading cluster ID for follower catalog manager: success
I20260807 07:52:24.937865 5657 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: acquiring CA information for follower catalog manager: success
I20260807 07:52:24.940442 5657 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:24.956957 5525 master_runner.cc:187] Error getting master registration for 127.2.169.251:34401: Network error: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111),
I20260807 07:52:24.960625 5525 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:25.238023 5641 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:25.238317 5641 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:25.238485 5641 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:25.247093 5641 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:25.247172 5641 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:25.247257 5641 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:25.247326 5641 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:25.257699 5641 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.2.169.251
--webserver_port=43555
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:25.259414 5641 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:25.262097 5641 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:25.271345 5665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:25.275053 5666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:25.276993 5668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:25.278012 5641 server_base.cc:1061] running on GCE node
I20260807 07:52:25.278975 5641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:25.282924 5641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:25.284211 5641 hybrid_clock.cc:648] HybridClock initialized: now 1786089145284155 us; error 44 us; skew 500 ppm
I20260807 07:52:25.284714 5641 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:25.287343 5641 webserver.cc:533] Webserver started at http://127.2.169.251:43555/ using document root <none> and password file <none>
I20260807 07:52:25.288151 5641 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:25.288300 5641 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:25.294698 5641 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 07:52:25.298238 5676 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:25.299927 5641 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 07:52:25.300133 5641 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:25.300905 5641 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:25.324091 5641 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:25.325229 5641 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:25.325560 5641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:25.354076 5641 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.251:34401
I20260807 07:52:25.354076 5747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.251:34401 every 8 connection(s)
I20260807 07:52:25.356041 5641 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
I20260807 07:52:25.356774 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 5641
I20260807 07:52:25.357601 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:36591
--local_ip_for_outbound_sockets=127.2.169.193
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589
--webserver_port=34149
--webserver_interface=127.2.169.193
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401 with env {}
I20260807 07:52:25.368934 5748 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:25.376905 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap starting.
I20260807 07:52:25.401949 5748 log.cc:826] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:25.412425 5755 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:25.412753 5755 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:25.415275 5755 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947), ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401)
I20260807 07:52:25.442731 5603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731" is_pre_election: true
I20260807 07:52:25.443301 5603 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 2.
I20260807 07:52:25.443169 5491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075" is_pre_election: true
I20260807 07:52:25.443729 5491 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 2.
I20260807 07:52:25.444296 5354 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 279e50b67caa47fb95d0c1e1beefd731, a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:25.445940 5755 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 07:52:25.446094 5755 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:25.446182 5755 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:25.451171 5755 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:25.452036 5755 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 3 election: Requested vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947), ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401)
I20260807 07:52:25.453403 5491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075"
I20260807 07:52:25.453630 5491 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:25.456269 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=21 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:25.456947 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap complete.
I20260807 07:52:25.462234 5491 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 3.
I20260807 07:52:25.464587 5603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731"
I20260807 07:52:25.464815 5603 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:25.465885 5748 raft_consensus.cc:359] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:25.466934 5748 raft_consensus.cc:740] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea850e24aa3845a08ce29d1510612bb7, State: Initialized, Role: FOLLOWER
I20260807 07:52:25.467811 5748 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:25.469715 5603 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 3.
I20260807 07:52:25.469260 5760 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.470104 5760 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.470259 5356 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 279e50b67caa47fb95d0c1e1beefd731, a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters:
I20260807 07:52:25.470743 5755 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:25.471098 5748 sys_catalog.cc:565] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:25.471163 5714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "ea850e24aa3845a08ce29d1510612bb7"
I20260807 07:52:25.471628 5714 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:25.471120 5715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 3 candidate_status { last_received { term: 2 index: 42 } } ignore_live_leader: false dest_uuid: "ea850e24aa3845a08ce29d1510612bb7" is_pre_election: true
I20260807 07:52:25.481635 5714 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 3.
I20260807 07:52:25.482511 5755 raft_consensus.cc:697] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 LEADER]: Becoming Leader. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Running, Role: LEADER
I20260807 07:52:25.483071 5755 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 2.42, Last appended by leader: 42, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } }
I20260807 07:52:25.486677 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.486975 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:25.487870 5772 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:25.491386 5772 catalog_manager.cc:679] Loaded metadata for table default.1_2 [id=44b956d4571840a780d714244fff640c]
I20260807 07:52:25.492093 5772 catalog_manager.cc:679] Loaded metadata for table default.0_5 [id=73efb1d25ad24e9bb7ad4fcd27db89e0]
I20260807 07:52:25.492681 5772 catalog_manager.cc:679] Loaded metadata for table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84]
I20260807 07:52:25.493312 5772 catalog_manager.cc:679] Loaded metadata for table default.0_4 [id=b198b5c57b4d4f89a279ae74df9fe9fc]
I20260807 07:52:25.493908 5772 catalog_manager.cc:679] Loaded metadata for table default.1_4 [id=d44dab2abaf647b0a118ad2af02204d0]
I20260807 07:52:25.494508 5772 catalog_manager.cc:679] Loaded metadata for table test-workload [id=e956c0fa574044c6b86ec65f8705a989]
I20260807 07:52:25.495024 5772 catalog_manager.cc:679] Loaded metadata for table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c]
I20260807 07:52:25.495609 5772 catalog_manager.cc:679] Loaded metadata for table default.1_5 [id=f2e18629f8db4f5f9e360d55d59d8526]
I20260807 07:52:25.496122 5772 catalog_manager.cc:679] Loaded metadata for table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e]
I20260807 07:52:25.498754 5772 tablet_loader.cc:96] loaded metadata for tablet 1298cbdc19e640b4bbbeeaf1c34d73bf (table default.1_5 [id=f2e18629f8db4f5f9e360d55d59d8526])
I20260807 07:52:25.499325 5772 tablet_loader.cc:96] loaded metadata for tablet 23de065eeead41b8aaec4866731369f5 (table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84])
I20260807 07:52:25.499788 5772 tablet_loader.cc:96] loaded metadata for tablet 7c30a1b53ff54e9a8515149bd3d5d84f (table test-workload [id=e956c0fa574044c6b86ec65f8705a989])
I20260807 07:52:25.500228 5772 tablet_loader.cc:96] loaded metadata for tablet ba7a33d575f4453dbf1b8861634a3f98 (table default.1_2 [id=44b956d4571840a780d714244fff640c])
I20260807 07:52:25.500682 5772 tablet_loader.cc:96] loaded metadata for tablet bcabb79073b040bfa91e1733e3d96128 (table default.0_4 [id=b198b5c57b4d4f89a279ae74df9fe9fc])
I20260807 07:52:25.501173 5772 tablet_loader.cc:96] loaded metadata for tablet de31825f9a984012abe2d537ea2b84f5 (table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e])
I20260807 07:52:25.501722 5772 tablet_loader.cc:96] loaded metadata for tablet dea58cd0bc524d3e842a2feb335a6bfe (table default.1_4 [id=d44dab2abaf647b0a118ad2af02204d0])
I20260807 07:52:25.502177 5772 tablet_loader.cc:96] loaded metadata for tablet e25a88e31cf744e8b38bc2996babb16a (table default.0_5 [id=73efb1d25ad24e9bb7ad4fcd27db89e0])
I20260807 07:52:25.502631 5772 tablet_loader.cc:96] loaded metadata for tablet fda7602334414a979800d942a24cfb86 (table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c])
I20260807 07:52:25.503026 5772 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:25.504635 5772 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:25.504771 5772 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:25.507318 5772 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:25.508953 5772 catalog_manager.cc:6081] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Loaded TSK: 0
I20260807 07:52:25.510252 5772 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:25.516047 5780 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:25.516175 5780 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: loading cluster ID for follower catalog manager: success
I20260807 07:52:25.520418 5780 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: acquiring CA information for follower catalog manager: success
I20260807 07:52:25.523237 5780 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:25.525015 5371 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53160:
name: "default.1_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:25.530164 5371 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53176:
name: "default.0_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 07:52:25.816892 5752 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:25.817322 5752 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:25.817495 5752 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:25.832778 5752 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:25.833031 5752 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:25.844295 5752 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:36591
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=34149
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:25.846031 5752 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:25.848134 5752 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:25.858534 5790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:25.858840 5791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:25.860405 5793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:25.861711 5752 server_base.cc:1061] running on GCE node
I20260807 07:52:25.862316 5752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:25.863345 5752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:25.864603 5752 hybrid_clock.cc:648] HybridClock initialized: now 1786089145864534 us; error 57 us; skew 500 ppm
I20260807 07:52:25.864986 5752 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:25.865617 5714 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 3 index: 43. (index mismatch)
I20260807 07:52:25.866698 5755 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260807 07:52:25.869156 5603 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 3 index: 43. (index mismatch)
I20260807 07:52:25.870169 5755 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260807 07:52:25.883091 5752 webserver.cc:533] Webserver started at http://127.2.169.193:34149/ using document root <none> and password file <none>
I20260807 07:52:25.883010 5760 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.883257 5760 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.884212 5752 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:25.884367 5752 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:25.887970 5800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.888283 5800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.893081 5491 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 2 index: 42. Preceding OpId from leader: term: 3 index: 43. (index mismatch)
I20260807 07:52:25.895506 5755 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260807 07:52:25.904970 5800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.905261 5800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.906886 5752 fs_manager.cc:714] Time spent opening directory manager: real 0.019s user 0.009s sys 0.009s
I20260807 07:52:25.911310 5760 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.911561 5760 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.912019 5755 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.912352 5755 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:25.912768 5755 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.913014 5755 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:25.919592 5815 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:25.921818 5813 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.922197 5813 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.922288 5752 fs_manager.cc:730] Time spent opening block manager: real 0.012s user 0.003s sys 0.007s
I20260807 07:52:25.922505 5813 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:25.922713 5813 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:25.922518 5752 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:25.923642 5752 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:25.959280 5752 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:25.960330 5752 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:25.960645 5752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:25.961831 5752 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:25.964490 5826 ts_tablet_manager.cc:548] Loading tablet metadata (0/9 complete)
I20260807 07:52:25.986243 5752 ts_tablet_manager.cc:585] Loaded tablet metadata (9 total tablets, 9 live tablets)
I20260807 07:52:25.986378 5752 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.022s user 0.001s sys 0.001s
I20260807 07:52:25.986547 5752 ts_tablet_manager.cc:600] Registering tablets (0/9 complete)
I20260807 07:52:25.993121 5826 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.026280 5752 ts_tablet_manager.cc:616] Registered 9 tablets
I20260807 07:52:26.026566 5752 ts_tablet_manager.cc:595] Time spent register tablets: real 0.040s user 0.019s sys 0.020s
I20260807 07:52:26.062244 5826 log.cc:826] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:26.069290 5826 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.070109 5826 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.071614 5826 ts_tablet_manager.cc:1403] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.079s user 0.021s sys 0.051s
I20260807 07:52:26.083333 5826 raft_consensus.cc:359] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.084105 5826 raft_consensus.cc:740] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.084944 5826 consensus_queue.cc:260] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.085141 5826 raft_consensus.cc:399] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.085302 5826 raft_consensus.cc:493] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.085556 5826 raft_consensus.cc:3060] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:26.093165 5826 raft_consensus.cc:515] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.093976 5826 leader_election.cc:304] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.096282 5826 leader_election.cc:290] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:26.097236 5939 raft_consensus.cc:2804] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:26.099447 5826 ts_tablet_manager.cc:1434] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.028s user 0.018s sys 0.010s
I20260807 07:52:26.101600 5939 raft_consensus.cc:697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.102077 5826 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.102800 5939 consensus_queue.cc:237] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.134462 5752 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:36591
I20260807 07:52:26.134678 5978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:36591 every 8 connection(s)
I20260807 07:52:26.139907 5752 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:26.147001 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 5752
I20260807 07:52:26.246568 5826 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.247255 5826 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.249990 5980 heartbeater.cc:344] Connected to a master server at 127.2.169.252:39947
I20260807 07:52:26.250102 5826 ts_tablet_manager.cc:1403] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.147s user 0.032s sys 0.081s
I20260807 07:52:26.250519 5980 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:26.251614 5826 raft_consensus.cc:359] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.251785 5980 heartbeater.cc:507] Master 127.2.169.252:39947 requested a full tablet report, sending...
I20260807 07:52:26.251876 5826 raft_consensus.cc:740] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.252238 5826 consensus_queue.cc:260] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.252460 5826 raft_consensus.cc:399] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.252596 5826 raft_consensus.cc:493] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.252712 5826 raft_consensus.cc:3060] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:26.254534 5981 heartbeater.cc:344] Connected to a master server at 127.2.169.254:45309
I20260807 07:52:26.254705 5981 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:26.255322 5981 heartbeater.cc:507] Master 127.2.169.254:45309 requested a full tablet report, sending...
I20260807 07:52:26.256644 5826 raft_consensus.cc:515] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.256942 5826 leader_election.cc:304] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.257035 5979 heartbeater.cc:344] Connected to a master server at 127.2.169.251:34401
I20260807 07:52:26.257148 5979 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:26.257261 5826 leader_election.cc:290] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:26.257855 5979 heartbeater.cc:507] Master 127.2.169.251:34401 requested a full tablet report, sending...
I20260807 07:52:26.258214 5942 raft_consensus.cc:2804] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:26.258396 5942 raft_consensus.cc:697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.258404 5585 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:26.258713 5942 consensus_queue.cc:237] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.261046 5982 heartbeater.cc:344] Connected to a master server at 127.2.169.253:45589
I20260807 07:52:26.261243 5982 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:26.261772 5982 heartbeater.cc:507] Master 127.2.169.253:45589 requested a full tablet report, sending...
I20260807 07:52:26.262706 5699 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:26.264477 5826 ts_tablet_manager.cc:1434] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260807 07:52:26.264822 5826 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.270183 5478 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:26.271054 5371 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:26.275164 5371 catalog_manager.cc:5697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.276206 5371 catalog_manager.cc:5697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.283005 5803 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } }
I20260807 07:52:26.283268 5826 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.283308 5803 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:26.283833 5826 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.285190 5826 ts_tablet_manager.cc:1403] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.008s
I20260807 07:52:26.286121 5826 raft_consensus.cc:359] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.286309 5826 raft_consensus.cc:740] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.286592 5826 consensus_queue.cc:260] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.286765 5826 raft_consensus.cc:399] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.286860 5826 raft_consensus.cc:493] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.286952 5826 raft_consensus.cc:3060] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:26.289839 5826 raft_consensus.cc:515] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.289997 5826 leader_election.cc:304] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.290211 5826 leader_election.cc:290] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:26.290251 5942 raft_consensus.cc:2804] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:26.290580 5942 raft_consensus.cc:697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.290714 5826 ts_tablet_manager.cc:1434] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.005s user 0.008s sys 0.001s
I20260807 07:52:26.290818 5942 consensus_queue.cc:237] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.291054 5826 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.299626 5371 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:48443
I20260807 07:52:26.303093 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:26.303509 5981 heartbeater.cc:499] Master 127.2.169.254:45309 was elected leader, sending a full tablet report...
I20260807 07:52:26.312620 5826 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.313489 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:26.313290 5826 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.314895 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.250:34449
--webserver_interface=127.2.169.250
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:26.315179 5826 ts_tablet_manager.cc:1403] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.019s
I20260807 07:52:26.316314 5826 raft_consensus.cc:359] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.316540 5826 raft_consensus.cc:740] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.316843 5826 consensus_queue.cc:260] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.317018 5826 raft_consensus.cc:399] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.317121 5826 raft_consensus.cc:493] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.317215 5826 raft_consensus.cc:3060] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:26.320333 5826 raft_consensus.cc:515] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.320575 5826 leader_election.cc:304] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.320809 5826 leader_election.cc:290] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:26.321296 5826 ts_tablet_manager.cc:1434] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260807 07:52:26.321314 5939 raft_consensus.cc:2804] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:26.321542 5939 raft_consensus.cc:697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.321658 5826 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.321847 5939 consensus_queue.cc:237] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.324558 5372 catalog_manager.cc:5697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.347739 5372 catalog_manager.cc:5697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.349720 5826 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.350323 5826 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.351572 5826 ts_tablet_manager.cc:1403] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.022s
I20260807 07:52:26.352746 5826 raft_consensus.cc:359] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.361411 5826 raft_consensus.cc:740] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.361756 5826 consensus_queue.cc:260] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.361924 5826 raft_consensus.cc:399] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.362015 5826 raft_consensus.cc:493] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.362224 5826 raft_consensus.cc:3060] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:26.365093 5826 raft_consensus.cc:515] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.365378 5826 leader_election.cc:304] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.365593 5826 leader_election.cc:290] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:26.365676 5942 raft_consensus.cc:2804] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:26.365834 5942 raft_consensus.cc:697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.366070 5826 ts_tablet_manager.cc:1434] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.014s user 0.000s sys 0.007s
I20260807 07:52:26.366101 5942 consensus_queue.cc:237] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.366374 5826 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.379338 5372 catalog_manager.cc:5697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.395247 5826 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.395830 5826 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.397154 5826 ts_tablet_manager.cc:1403] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.015s
I20260807 07:52:26.397989 5826 raft_consensus.cc:359] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.398186 5826 raft_consensus.cc:740] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.398469 5826 consensus_queue.cc:260] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.398653 5826 raft_consensus.cc:399] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.398763 5826 raft_consensus.cc:493] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.398871 5826 raft_consensus.cc:3060] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:26.401835 5826 raft_consensus.cc:515] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.402098 5826 leader_election.cc:304] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.402379 5942 raft_consensus.cc:2804] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:26.402577 5942 raft_consensus.cc:697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.402832 5942 consensus_queue.cc:237] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.403338 5826 leader_election.cc:290] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:26.403927 5826 ts_tablet_manager.cc:1434] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.007s user 0.007s sys 0.002s
I20260807 07:52:26.405807 5826 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.406804 5372 catalog_manager.cc:5697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.427464 5826 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.428021 5826 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.429069 5826 ts_tablet_manager.cc:1403] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.011s
I20260807 07:52:26.429858 5826 raft_consensus.cc:359] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.430034 5826 raft_consensus.cc:740] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.430280 5826 consensus_queue.cc:260] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.430428 5826 raft_consensus.cc:399] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.430538 5826 raft_consensus.cc:493] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.430645 5826 raft_consensus.cc:3060] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:26.432667 5826 raft_consensus.cc:515] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.432888 5826 leader_election.cc:304] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.433110 5826 leader_election.cc:290] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:26.433205 5942 raft_consensus.cc:2804] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:26.433411 5942 raft_consensus.cc:697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.433684 5826 ts_tablet_manager.cc:1434] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260807 07:52:26.433727 5942 consensus_queue.cc:237] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.433951 5826 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.444906 5372 catalog_manager.cc:5697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.449225 5826 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.449734 5826 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.450961 5826 ts_tablet_manager.cc:1403] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.010s
I20260807 07:52:26.451766 5826 raft_consensus.cc:359] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.451947 5826 raft_consensus.cc:740] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.452278 5826 consensus_queue.cc:260] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.452450 5826 raft_consensus.cc:399] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.452539 5826 raft_consensus.cc:493] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.452625 5826 raft_consensus.cc:3060] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:26.456264 5826 raft_consensus.cc:515] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.456539 5826 leader_election.cc:304] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.456812 5826 leader_election.cc:290] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:26.456861 5942 raft_consensus.cc:2804] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:26.457314 5826 ts_tablet_manager.cc:1434] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260807 07:52:26.457603 5942 raft_consensus.cc:697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.457880 5942 consensus_queue.cc:237] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.457661 5826 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.462746 5372 catalog_manager.cc:5697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.477833 5826 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:26.478394 5826 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:26.479794 5826 ts_tablet_manager.cc:1403] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.005s
I20260807 07:52:26.480736 5826 raft_consensus.cc:359] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.480959 5826 raft_consensus.cc:740] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.481278 5826 consensus_queue.cc:260] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.481487 5826 raft_consensus.cc:399] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.481596 5826 raft_consensus.cc:493] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.481689 5826 raft_consensus.cc:3060] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:26.484529 5826 raft_consensus.cc:515] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.484779 5826 leader_election.cc:304] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.485005 5826 leader_election.cc:290] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:26.485128 5942 raft_consensus.cc:2804] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:26.485368 5942 raft_consensus.cc:697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.485697 5826 ts_tablet_manager.cc:1434] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 07:52:26.485707 5942 consensus_queue.cc:237] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.489691 5372 catalog_manager.cc:5697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.557492 5371 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53226:
name: "default.0_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:26.595873 5371 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53236:
name: "default.1_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:26.627506 5888 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce0b4ae485c453aaab7a4c0ec4a75e2 (DEFAULT_TABLE table=default.1_7 [id=178ea3a0eda140daaf336ed4e4e4fb75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:26.627768 5889 tablet_service.cc:1511] Processing CreateTablet for tablet 000ff4da6ee741c68edcce5bebf25460 (DEFAULT_TABLE table=default.0_7 [id=be1ad2f73a4f4c79a31a9874293ea8b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:26.628679 5888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce0b4ae485c453aaab7a4c0ec4a75e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:26.628955 5889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000ff4da6ee741c68edcce5bebf25460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:26.638424 5826 tablet_bootstrap.cc:492] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.640885 5826 tablet_bootstrap.cc:654] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:26.643801 5826 tablet_bootstrap.cc:492] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:26.643970 5826 ts_tablet_manager.cc:1403] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:52:26.644842 5826 raft_consensus.cc:359] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.645037 5826 raft_consensus.cc:385] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:26.645143 5826 raft_consensus.cc:740] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.645444 5826 consensus_queue.cc:260] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.645655 5826 raft_consensus.cc:399] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.645740 5826 raft_consensus.cc:493] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.645859 5826 raft_consensus.cc:3060] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:26.648906 5826 raft_consensus.cc:515] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.649187 5826 leader_election.cc:304] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.649454 5826 leader_election.cc:290] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:26.649587 5942 raft_consensus.cc:2804] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:26.649772 5942 raft_consensus.cc:697] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.650005 5942 consensus_queue.cc:237] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.650312 5826 ts_tablet_manager.cc:1434] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260807 07:52:26.650614 5826 tablet_bootstrap.cc:492] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:26.652586 5826 tablet_bootstrap.cc:654] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:26.653412 5371 catalog_manager.cc:5697] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:26.669399 5826 tablet_bootstrap.cc:492] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:26.669653 5826 ts_tablet_manager.cc:1403] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20260807 07:52:26.671072 5826 raft_consensus.cc:359] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.671404 5826 raft_consensus.cc:385] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:26.671516 5826 raft_consensus.cc:740] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:26.671851 5826 consensus_queue.cc:260] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.672134 5826 raft_consensus.cc:399] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:26.672328 5826 raft_consensus.cc:493] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:26.672519 5826 raft_consensus.cc:3060] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:26.676952 5826 raft_consensus.cc:515] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.677320 5826 leader_election.cc:304] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:26.677736 5826 leader_election.cc:290] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:26.677801 5942 raft_consensus.cc:2804] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:26.678115 5942 raft_consensus.cc:697] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:26.678565 5942 consensus_queue.cc:237] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:26.679852 5826 ts_tablet_manager.cc:1434] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260807 07:52:26.687528 5371 catalog_manager.cc:5697] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
W20260807 07:52:26.890714 6007 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:26.891119 6007 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:26.891274 6007 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:26.901830 6007 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:26.901975 6007 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:26.902122 6007 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:26.902230 6007 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:26.915505 6007 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.250:34449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
--webserver_interface=127.2.169.250
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:26.917378 6007 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:26.919394 6007 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:26.935819 6055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:26.936278 6056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:26.937064 6060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:26.938122 6007 server_base.cc:1061] running on GCE node
I20260807 07:52:26.938988 6007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:26.940645 6007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:26.941999 6007 hybrid_clock.cc:648] HybridClock initialized: now 1786089146941904 us; error 55 us; skew 500 ppm
I20260807 07:52:26.942461 6007 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:26.945142 6007 webserver.cc:533] Webserver started at http://127.2.169.250:42927/ using document root <none> and password file <none>
I20260807 07:52:26.946059 6007 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:26.946218 6007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:26.946681 6007 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:26.949208 6007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/instance:
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:26.950186 6007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal/instance:
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:26.955813 6007 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 07:52:26.959879 6068 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:26.961292 6007 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 07:52:26.961575 6007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:26.961827 6007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:27.000321 6007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:27.001715 6007 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:27.002168 6007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:27.037621 6007 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.250:34449
I20260807 07:52:27.037669 6129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.250:34449 every 8 connection(s)
I20260807 07:52:27.039422 6007 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
I20260807 07:52:27.043635 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6007
I20260807 07:52:27.045117 6130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:27.053820 6130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:27.070330 6130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:27.078341 6130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:27.091230 6130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:27.099804 6130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:27.115952 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap starting.
I20260807 07:52:27.120250 6130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:27.122033 6130 log.cc:826] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:27.126202 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: No bootstrap required, opened a new log
I20260807 07:52:27.132975 6130 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } }
I20260807 07:52:27.133389 6130 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:27.133566 6130 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7e184f10af48e9a62e962cf40db93a, State: Initialized, Role: FOLLOWER
I20260807 07:52:27.134315 6130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } }
I20260807 07:52:27.135524 6137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } } }
I20260807 07:52:27.136058 6137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.136618 6130 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:27.152449 6150 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:27.152632 6150 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:27.164844 6007 master_runner.cc:429] Detected that this master 8a7e184f10af48e9a62e962cf40db93a is joining an existing cluster
I20260807 07:52:27.165004 6007 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260807 07:52:27.166198 6007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:52:27.166627 6007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 07:52:27.166854 6007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: stopping tablet replica
I20260807 07:52:27.169000 6007 master_runner.cc:434] Initiating AddMaster RPC to add 127.2.169.250:34449
I20260807 07:52:27.253927 5372 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.2.169.250:34449: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a"
member_type: NON_VOTER
last_known_addr {
host: "127.2.169.250"
port: 34449
}
attrs {
promote: true
}
}
dest_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
cas_config_opid_index: 36
I20260807 07:52:27.255517 5372 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 58, Committed index: 58, Last appended: 3.58, Last appended by leader: 42, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } }
I20260807 07:52:27.258806 5714 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 58. Preceding OpId from leader: term: 3 index: 59. (index mismatch)
I20260807 07:52:27.259537 5603 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 58. Preceding OpId from leader: term: 3 index: 59. (index mismatch)
I20260807 07:52:27.259593 5491 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 58. Preceding OpId from leader: term: 3 index: 59. (index mismatch)
W20260807 07:52:27.259804 5353 consensus_peers.cc:597] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a -> Peer 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449): Couldn't send request to peer 8a7e184f10af48e9a62e962cf40db93a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:27.260418 5762 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20260807 07:52:27.261245 5762 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20260807 07:52:27.261650 6166 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20260807 07:52:27.265079 5762 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 LEADER]: Committing config change with OpId 3.59: config changed from index 36 to 59, NON_VOTER 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.265733 5714 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Committing config change with OpId 3.59: config changed from index 36 to 59, NON_VOTER 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.266148 5603 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Committing config change with OpId 3.59: config changed from index 36 to 59, NON_VOTER 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.272487 5762 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.2.169.250:34449
I20260807 07:52:27.273412 6007 master.cc:562] Master@127.2.169.250:34449 shutting down...
I20260807 07:52:27.276291 5491 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Committing config change with OpId 3.59: config changed from index 36 to 59, NON_VOTER 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) added. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.282060 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.282104 6166 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.282318 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.282346 6166 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.282781 6165 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.282984 6166 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.283149 6165 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.283218 6166 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.283981 6172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.284406 6172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.287339 6174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.287675 6174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.288247 6169 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.288664 6169 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.326486 6007 master.cc:584] Master@127.2.169.250:34449 shutdown complete.
I20260807 07:52:27.326606 6007 master_runner.cc:306] Clearing existing system tablet
I20260807 07:52:27.327062 6007 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:27.327176 6007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:27.330225 6007 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 07:52:27.336261 6178 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:27.336733 6007 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:27.336985 6007 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.002s sys 0.004s
I20260807 07:52:27.337085 6007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:27.340648 6007 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 07:52:27.346385 6007 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 07:52:27.346539 6007 log.cc:1199] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Deleting WAL directory at /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal/wals/00000000000000000000000000000000
I20260807 07:52:27.347110 6007 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Deleting consensus metadata
I20260807 07:52:27.347396 6007 master_runner.cc:316] Copying system tablet from 127.2.169.254:45309
I20260807 07:52:27.351409 6007 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: tablet copy: Beginning tablet copy session from remote peer at address 127.2.169.254:45309
I20260807 07:52:27.362092 5399 tablet_copy_service.cc:140] P ca5ccffd14db4e15be5ec8b65fc2502a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8a7e184f10af48e9a62e962cf40db93a ({username='slave'} at 127.0.0.1:53268)
I20260807 07:52:27.362449 5399 tablet_copy_service.cc:161] P ca5ccffd14db4e15be5ec8b65fc2502a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8a7e184f10af48e9a62e962cf40db93a at {username='slave'} at 127.0.0.1:53268: session id = 8a7e184f10af48e9a62e962cf40db93a-00000000000000000000000000000000
I20260807 07:52:27.366277 5399 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 07:52:27.368535 6007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 07:52:27.377789 6007 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: tablet copy: Starting download of 0 data blocks...
I20260807 07:52:27.378178 6007 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: tablet copy: Starting download of 1 WAL segments...
W20260807 07:52:27.379755 5372 master.cc:680] Network error: unable to get registration information for peer 8a7e184f10af48e9a62e962cf40db93a: Client connection negotiation failed: client connection to 127.2.169.250:34449: connect: Connection refused (error 111)
I20260807 07:52:27.382987 6007 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 07:52:27.387247 5399 tablet_copy_service.cc:342] P ca5ccffd14db4e15be5ec8b65fc2502a: Request end of tablet copy session 8a7e184f10af48e9a62e962cf40db93a-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:53268
I20260807 07:52:27.387727 5399 tablet_copy_service.cc:434] P ca5ccffd14db4e15be5ec8b65fc2502a: ending tablet copy session 8a7e184f10af48e9a62e962cf40db93a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8a7e184f10af48e9a62e962cf40db93a
W20260807 07:52:27.395551 6007 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260807 07:52:27.399192 6007 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:27.402383 6186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:27.403774 6185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:27.404870 6189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:27.405138 6007 server_base.cc:1061] running on GCE node
I20260807 07:52:27.405539 6007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:27.406158 6007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:27.407466 6007 hybrid_clock.cc:648] HybridClock initialized: now 1786089147407267 us; error 92 us; skew 500 ppm
I20260807 07:52:27.407752 6007 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:27.409246 6007 webserver.cc:533] Webserver started at http://127.2.169.250:36191/ using document root <none> and password file <none>
I20260807 07:52:27.409687 6007 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:27.409940 6007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:27.412803 6007 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 07:52:27.415475 6196 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:27.416322 6007 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 07:52:27.416595 6007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:27.416872 6007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:27.429106 6007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:27.430496 6007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:27.485633 6007 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.250:34449
I20260807 07:52:27.485734 6263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.250:34449 every 8 connection(s)
I20260807 07:52:27.487056 6007 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
I20260807 07:52:27.492115 6264 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:27.495049 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap starting.
I20260807 07:52:27.561396 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap replayed 1/1 log segments. Stats: ops{read=59 overwritten=0 applied=59 ignored=0} inserts{seen=25 ignored=0} mutations{seen=41 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:27.562115 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap complete.
I20260807 07:52:27.563457 6264 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } }
I20260807 07:52:27.563787 6264 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 8a7e184f10af48e9a62e962cf40db93a, State: Initialized, Role: LEARNER
I20260807 07:52:27.564199 6264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 59, Last appended: 3.59, Last appended by leader: 59, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } }
I20260807 07:52:27.566762 6264 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:27.566366 6268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.567173 6268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:27.586500 6279 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:27.586601 6279 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: loading cluster ID for follower catalog manager: success
I20260807 07:52:27.590204 6279 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: acquiring CA information for follower catalog manager: success
I20260807 07:52:27.593629 6279 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:27.692292 5372 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53278:
name: "default.1_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:27.709311 6234 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 LEARNER]: Deduplicated request from leader. Original: 3.58->[3.59-3.60] Dedup: 3.59->[3.60-3.60]
I20260807 07:52:27.743247 5889 tablet_service.cc:1511] Processing CreateTablet for tablet 5036daa1e5ed493db4a152f70bae715b (DEFAULT_TABLE table=default.1_8 [id=4552f125ce6649938ecd3b11c6da1ab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:27.743445 6268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.743818 6268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: LEARNER
I20260807 07:52:27.743851 5889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5036daa1e5ed493db4a152f70bae715b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:27.756413 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: NON_VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: true } } }
I20260807 07:52:27.756793 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.776265 6313 tablet_bootstrap.cc:492] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:27.787374 6173 raft_consensus.cc:1064] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: attempting to promote NON_VOTER 8a7e184f10af48e9a62e962cf40db93a to VOTER
I20260807 07:52:27.789359 6313 tablet_bootstrap.cc:654] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:27.789570 6173 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 61, Committed index: 61, Last appended: 3.61, Last appended by leader: 42, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:27.796387 5714 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 61. Preceding OpId from leader: term: 3 index: 62. (index mismatch)
I20260807 07:52:27.797147 5762 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20260807 07:52:27.797415 6233 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 LEARNER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 61. Preceding OpId from leader: term: 3 index: 62. (index mismatch)
I20260807 07:52:27.798267 6175 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20260807 07:52:27.800709 5603 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 61. Preceding OpId from leader: term: 3 index: 62. (index mismatch)
I20260807 07:52:27.801517 5490 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 61. Preceding OpId from leader: term: 3 index: 62. (index mismatch)
I20260807 07:52:27.801481 5762 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20260807 07:52:27.802287 6173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20260807 07:52:27.804821 6175 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 LEADER]: Committing config change with OpId 3.62: config changed from index 59 to 62, 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) changed from NON_VOTER to VOTER. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.809300 5603 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Committing config change with OpId 3.62: config changed from index 59 to 62, 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) changed from NON_VOTER to VOTER. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.823041 5490 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Committing config change with OpId 3.62: config changed from index 59 to 62, 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) changed from NON_VOTER to VOTER. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.828450 6313 tablet_bootstrap.cc:492] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:27.828750 6313 ts_tablet_manager.cc:1403] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.006s
I20260807 07:52:27.828652 6233 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 FOLLOWER]: Committing config change with OpId 3.62: config changed from index 59 to 62, 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) changed from NON_VOTER to VOTER. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.830416 6313 raft_consensus.cc:359] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.830526 6173 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.830713 6313 raft_consensus.cc:385] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:27.830911 6173 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.830926 6313 raft_consensus.cc:740] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:27.831257 6313 consensus_queue.cc:260] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.831641 6313 raft_consensus.cc:399] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:27.831830 6313 raft_consensus.cc:493] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:27.832127 6313 raft_consensus.cc:3060] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:27.832161 6173 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.832592 6173 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.832332 5714 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Committing config change with OpId 3.62: config changed from index 59 to 62, 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250) changed from NON_VOTER to VOTER. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.835951 6313 raft_consensus.cc:515] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.836300 6313 leader_election.cc:304] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:27.836262 6268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.836568 6268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.840193 5372 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53280:
name: "default.0_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:27.850767 6323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.851174 6323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.852130 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.852449 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.853021 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.853380 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.861505 6321 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.862897 6320 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.863385 6320 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.863925 6321 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:27.864533 6313 leader_election.cc:290] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:27.865154 6319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:27.865825 6322 raft_consensus.cc:2804] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:27.866346 6319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:27.869936 6322 raft_consensus.cc:697] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:27.870594 6322 consensus_queue.cc:237] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.872231 6313 ts_tablet_manager.cc:1434] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.043s user 0.010s sys 0.010s
I20260807 07:52:27.893747 5889 tablet_service.cc:1511] Processing CreateTablet for tablet 6e479075a88046939401633792a640aa (DEFAULT_TABLE table=default.0_8 [id=d1237537cd994e409252accbea3db924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:27.893695 5372 catalog_manager.cc:5697] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:27.894228 5889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e479075a88046939401633792a640aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:27.902130 6313 tablet_bootstrap.cc:492] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:27.905529 6313 tablet_bootstrap.cc:654] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:27.915126 6313 tablet_bootstrap.cc:492] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: No bootstrap required, opened a new log
I20260807 07:52:27.915525 6313 ts_tablet_manager.cc:1403] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20260807 07:52:27.916733 6313 raft_consensus.cc:359] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.916996 6313 raft_consensus.cc:385] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:27.917111 6313 raft_consensus.cc:740] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:27.917433 6313 consensus_queue.cc:260] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.917682 6313 raft_consensus.cc:399] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:27.917845 6313 raft_consensus.cc:493] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:27.917986 6313 raft_consensus.cc:3060] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:27.921839 6313 raft_consensus.cc:515] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.922115 6313 leader_election.cc:304] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:27.922477 6313 leader_election.cc:290] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:27.922508 6322 raft_consensus.cc:2804] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:27.922838 6322 raft_consensus.cc:697] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:27.923063 6322 consensus_queue.cc:237] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:27.924124 6313 ts_tablet_manager.cc:1434] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 07:52:27.930205 5372 catalog_manager.cc:5697] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:28.311969 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 5752
I20260807 07:52:28.323230 5974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:28.437881 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 5752
I20260807 07:52:28.468700 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 5316
I20260807 07:52:28.470054 5410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:28.593686 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 5316
W20260807 07:52:28.625027 5681 connection.cc:573] server connection from 127.0.0.1:45898 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 07:52:28.625756 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 5417
W20260807 07:52:28.625756 5572 connection.cc:573] server connection from 127.0.0.1:44968 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 07:52:28.627014 5515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:28.742268 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 5417
I20260807 07:52:28.773736 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 5525
I20260807 07:52:28.775036 5630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:28.887531 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 5525
I20260807 07:52:28.919312 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 5641
I20260807 07:52:28.920517 5742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:29.037618 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 5641
I20260807 07:52:29.064677 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6007
I20260807 07:52:29.065771 6259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:29.187505 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6007
I20260807 07:52:29.219417 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--webserver_interface=127.2.169.254
--webserver_port=40051
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
W20260807 07:52:29.601616 6367 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:29.601971 6367 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:29.602061 6367 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:29.612711 6367 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:29.612823 6367 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:29.612895 6367 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:29.612957 6367 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:29.623631 6367 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=40051
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:29.625492 6367 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:29.627568 6367 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:29.636430 6374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:29.636916 6373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:29.637296 6378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:29.638551 6367 server_base.cc:1061] running on GCE node
I20260807 07:52:29.639576 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:29.641041 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:29.642326 6367 hybrid_clock.cc:648] HybridClock initialized: now 1786089149642269 us; error 48 us; skew 500 ppm
I20260807 07:52:29.642755 6367 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:29.645569 6367 webserver.cc:533] Webserver started at http://127.2.169.254:40051/ using document root <none> and password file <none>
I20260807 07:52:29.646341 6367 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:29.646441 6367 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:29.651974 6367 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 07:52:29.654989 6386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:29.656361 6367 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 07:52:29.656533 6367 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "ca5ccffd14db4e15be5ec8b65fc2502a"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:29.657282 6367 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:29.687433 6367 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:29.688321 6367 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:29.688647 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:29.710124 6367 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:45309
I20260807 07:52:29.710175 6446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:45309 every 8 connection(s)
I20260807 07:52:29.712284 6367 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:29.716581 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6367
I20260807 07:52:29.717422 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--webserver_interface=127.2.169.253
--webserver_port=41101
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:29.722220 6448 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:29.727677 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap starting.
I20260807 07:52:29.758051 6448 log.cc:826] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:29.808990 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap replayed 1/1 log segments. Stats: ops{read=67 overwritten=0 applied=67 ignored=0} inserts{seen=29 ignored=0} mutations{seen=46 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:29.809696 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Bootstrap complete.
I20260807 07:52:29.815796 6448 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:29.816483 6448 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Initialized, Role: FOLLOWER
I20260807 07:52:29.817102 6448 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:29.818109 6454 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:29.818468 6454 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:29.819238 6448 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:29.842079 6467 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:29.842293 6467 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: loading cluster ID for follower catalog manager: success
I20260807 07:52:29.840128 6367 master_runner.cc:187] Error getting master registration for 127.2.169.253:45589: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45589: connect: Connection refused (error 111),
I20260807 07:52:29.846408 6367 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260807 07:52:29.846975 6467 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: acquiring CA information for follower catalog manager: success
I20260807 07:52:29.850139 6467 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 07:52:30.103193 6452 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:30.103533 6452 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:30.103667 6452 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:30.112080 6452 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:30.112171 6452 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:30.112255 6452 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:30.112351 6452 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:30.122776 6452 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=41101
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:30.124521 6452 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:30.126485 6452 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:30.134581 6476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:30.135778 6477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:30.137133 6479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:30.138144 6452 server_base.cc:1061] running on GCE node
I20260807 07:52:30.139139 6452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:30.141112 6452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:30.142406 6452 hybrid_clock.cc:648] HybridClock initialized: now 1786089150142318 us; error 100 us; skew 500 ppm
I20260807 07:52:30.142858 6452 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:30.145920 6452 webserver.cc:533] Webserver started at http://127.2.169.253:41101/ using document root <none> and password file <none>
I20260807 07:52:30.146987 6452 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:30.147130 6452 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:30.155726 6452 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 07:52:30.159744 6486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:30.161639 6452 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 07:52:30.161835 6452 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "a486fca3ed6e467daa9af0e316f2e075"
format_stamp: "Formatted at 2026-08-07 07:52:14 on dist-test-slave-bw88"
I20260807 07:52:30.162896 6452 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:30.190877 6452 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:30.192165 6452 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:30.192530 6452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:30.223079 6452 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:45589
I20260807 07:52:30.223079 6554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:45589 every 8 connection(s)
I20260807 07:52:30.224972 6452 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:30.225423 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6452
I20260807 07:52:30.226261 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--webserver_interface=127.2.169.252
--webserver_port=46357
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:30.238163 6555 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:30.242209 6555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap starting.
I20260807 07:52:30.286038 6555 log.cc:826] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:30.368382 6555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap replayed 1/1 log segments. Stats: ops{read=67 overwritten=0 applied=67 ignored=0} inserts{seen=29 ignored=0} mutations{seen=46 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:30.369241 6555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: Bootstrap complete.
I20260807 07:52:30.379139 6555 raft_consensus.cc:359] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:30.380239 6555 raft_consensus.cc:740] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: a486fca3ed6e467daa9af0e316f2e075, State: Initialized, Role: FOLLOWER
I20260807 07:52:30.381106 6555 consensus_queue.cc:260] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:30.382501 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:30.382854 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:30.383908 6555 sys_catalog.cc:565] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:30.408622 6578 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:30.408811 6578 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: loading cluster ID for follower catalog manager: success
I20260807 07:52:30.413717 6578 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: acquiring CA information for follower catalog manager: success
I20260807 07:52:30.416842 6578 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:30.426499 6452 master_runner.cc:187] Error getting master registration for 127.2.169.252:39947: Network error: Client connection negotiation failed: client connection to 127.2.169.252:39947: connect: Connection refused (error 111),
I20260807 07:52:30.430418 6452 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:30.751312 6558 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:30.751770 6558 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:30.751928 6558 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:30.765880 6558 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:30.766024 6558 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:30.766131 6558 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:30.766209 6558 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:30.783442 6558 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=46357
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:30.785789 6558 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:30.788396 6558 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:30.800922 6587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:30.801781 6588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:30.802726 6590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:30.803956 6558 server_base.cc:1061] running on GCE node
I20260807 07:52:30.804978 6558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:30.806826 6558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:30.808085 6558 hybrid_clock.cc:648] HybridClock initialized: now 1786089150808038 us; error 100 us; skew 500 ppm
I20260807 07:52:30.808554 6558 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:30.811448 6558 webserver.cc:533] Webserver started at http://127.2.169.252:46357/ using document root <none> and password file <none>
I20260807 07:52:30.812453 6558 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:30.812604 6558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:30.820232 6558 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 07:52:30.824209 6597 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:30.826012 6558 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 07:52:30.826216 6558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "279e50b67caa47fb95d0c1e1beefd731"
format_stamp: "Formatted at 2026-08-07 07:52:16 on dist-test-slave-bw88"
I20260807 07:52:30.827306 6558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:30.866202 6558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:30.867455 6558 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:30.867861 6558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:30.895053 6657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:39947 every 8 connection(s)
I20260807 07:52:30.895105 6558 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:39947
I20260807 07:52:30.897655 6558 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:30.905261 6658 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:30.905970 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6558
I20260807 07:52:30.906829 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--webserver_interface=127.2.169.251
--webserver_port=43555
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:30.909189 6658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap starting.
I20260807 07:52:30.961781 6664 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:30.963982 6658 log.cc:826] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:30.964411 6664 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:30.995522 6523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 4 candidate_status { last_received { term: 3 index: 67 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075" is_pre_election: true
I20260807 07:52:30.996273 6523 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 3.
I20260807 07:52:30.997615 6664 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947), ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401), 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449)
W20260807 07:52:30.997227 6392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111)
W20260807 07:52:31.006500 6389 leader_election.cc:336] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449): Network error: Client connection negotiation failed: client connection to 127.2.169.250:34449: connect: Connection refused (error 111)
W20260807 07:52:31.008813 6392 leader_election.cc:336] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401): Network error: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111)
I20260807 07:52:31.062529 6658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap replayed 1/1 log segments. Stats: ops{read=67 overwritten=0 applied=67 ignored=0} inserts{seen=29 ignored=0} mutations{seen=46 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:31.063324 6658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: Bootstrap complete.
I20260807 07:52:31.072619 6658 raft_consensus.cc:359] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.073618 6658 raft_consensus.cc:740] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e50b67caa47fb95d0c1e1beefd731, State: Initialized, Role: FOLLOWER
I20260807 07:52:31.074383 6658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.075627 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.076254 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.076797 6658 sys_catalog.cc:565] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:31.076855 6630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 4 candidate_status { last_received { term: 3 index: 67 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731" is_pre_election: true
I20260807 07:52:31.077452 6630 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 3.
I20260807 07:52:31.078368 6392 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 5 voters: 3 yes votes; 2 no votes. yes voters: 279e50b67caa47fb95d0c1e1beefd731, a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters: 8a7e184f10af48e9a62e962cf40db93a, ea850e24aa3845a08ce29d1510612bb7
I20260807 07:52:31.079337 6664 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 07:52:31.079507 6664 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:31.079605 6664 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:31.084709 6664 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.092264 6523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 4 candidate_status { last_received { term: 3 index: 67 } } ignore_live_leader: false dest_uuid: "a486fca3ed6e467daa9af0e316f2e075"
I20260807 07:52:31.092491 6523 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:31.094066 6664 leader_election.cc:290] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 election: Requested vote from peers a486fca3ed6e467daa9af0e316f2e075 (127.2.169.253:45589), 279e50b67caa47fb95d0c1e1beefd731 (127.2.169.252:39947), ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401), 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449)
I20260807 07:52:31.096340 6630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" candidate_term: 4 candidate_status { last_received { term: 3 index: 67 } } ignore_live_leader: false dest_uuid: "279e50b67caa47fb95d0c1e1beefd731"
I20260807 07:52:31.096586 6630 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 3 FOLLOWER]: Advancing to term 4
W20260807 07:52:31.097007 6389 leader_election.cc:336] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449): Network error: Client connection negotiation failed: client connection to 127.2.169.250:34449: connect: Connection refused (error 111)
I20260807 07:52:31.103575 6630 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 4.
I20260807 07:52:31.106179 6523 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ccffd14db4e15be5ec8b65fc2502a in term 4.
W20260807 07:52:31.106393 6392 leader_election.cc:336] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ea850e24aa3845a08ce29d1510612bb7 (127.2.169.251:34401): Network error: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111)
I20260807 07:52:31.107546 6390 leader_election.cc:304] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 5 voters: 3 yes votes; 2 no votes. yes voters: 279e50b67caa47fb95d0c1e1beefd731, a486fca3ed6e467daa9af0e316f2e075, ca5ccffd14db4e15be5ec8b65fc2502a; no voters: 8a7e184f10af48e9a62e962cf40db93a, ea850e24aa3845a08ce29d1510612bb7
I20260807 07:52:31.108676 6664 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:31.109277 6664 raft_consensus.cc:697] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [term 4 LEADER]: Becoming Leader. State: Replica: ca5ccffd14db4e15be5ec8b65fc2502a, State: Running, Role: LEADER
I20260807 07:52:31.109894 6664 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 67, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.113003 6680 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.113271 6680 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:31.115352 6684 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:31.118099 6684 catalog_manager.cc:679] Loaded metadata for table default.1_7 [id=178ea3a0eda140daaf336ed4e4e4fb75]
I20260807 07:52:31.118602 6684 catalog_manager.cc:679] Loaded metadata for table default.1_2 [id=44b956d4571840a780d714244fff640c]
I20260807 07:52:31.118971 6684 catalog_manager.cc:679] Loaded metadata for table default.1_8 [id=4552f125ce6649938ecd3b11c6da1ab3]
I20260807 07:52:31.119313 6684 catalog_manager.cc:679] Loaded metadata for table default.0_5 [id=73efb1d25ad24e9bb7ad4fcd27db89e0]
I20260807 07:52:31.119629 6684 catalog_manager.cc:679] Loaded metadata for table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84]
I20260807 07:52:31.119999 6684 catalog_manager.cc:679] Loaded metadata for table default.0_4 [id=b198b5c57b4d4f89a279ae74df9fe9fc]
I20260807 07:52:31.120329 6684 catalog_manager.cc:679] Loaded metadata for table default.0_7 [id=be1ad2f73a4f4c79a31a9874293ea8b5]
I20260807 07:52:31.120787 6684 catalog_manager.cc:679] Loaded metadata for table default.0_8 [id=d1237537cd994e409252accbea3db924]
I20260807 07:52:31.121130 6684 catalog_manager.cc:679] Loaded metadata for table default.1_4 [id=d44dab2abaf647b0a118ad2af02204d0]
I20260807 07:52:31.121712 6684 catalog_manager.cc:679] Loaded metadata for table test-workload [id=e956c0fa574044c6b86ec65f8705a989]
I20260807 07:52:31.122117 6684 catalog_manager.cc:679] Loaded metadata for table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c]
I20260807 07:52:31.122689 6684 catalog_manager.cc:679] Loaded metadata for table default.1_5 [id=f2e18629f8db4f5f9e360d55d59d8526]
I20260807 07:52:31.123106 6684 catalog_manager.cc:679] Loaded metadata for table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e]
I20260807 07:52:31.125433 6684 tablet_loader.cc:96] loaded metadata for tablet 000ff4da6ee741c68edcce5bebf25460 (table default.0_7 [id=be1ad2f73a4f4c79a31a9874293ea8b5])
I20260807 07:52:31.126020 6684 tablet_loader.cc:96] loaded metadata for tablet 1298cbdc19e640b4bbbeeaf1c34d73bf (table default.1_5 [id=f2e18629f8db4f5f9e360d55d59d8526])
I20260807 07:52:31.126468 6684 tablet_loader.cc:96] loaded metadata for tablet 23de065eeead41b8aaec4866731369f5 (table default.1_1 [id=866b717a97bd4b998a6fbb7e605a1f84])
I20260807 07:52:31.126917 6684 tablet_loader.cc:96] loaded metadata for tablet 3ce0b4ae485c453aaab7a4c0ec4a75e2 (table default.1_7 [id=178ea3a0eda140daaf336ed4e4e4fb75])
I20260807 07:52:31.127379 6684 tablet_loader.cc:96] loaded metadata for tablet 5036daa1e5ed493db4a152f70bae715b (table default.1_8 [id=4552f125ce6649938ecd3b11c6da1ab3])
I20260807 07:52:31.127801 6684 tablet_loader.cc:96] loaded metadata for tablet 6e479075a88046939401633792a640aa (table default.0_8 [id=d1237537cd994e409252accbea3db924])
I20260807 07:52:31.128232 6684 tablet_loader.cc:96] loaded metadata for tablet 7c30a1b53ff54e9a8515149bd3d5d84f (table test-workload [id=e956c0fa574044c6b86ec65f8705a989])
I20260807 07:52:31.128667 6684 tablet_loader.cc:96] loaded metadata for tablet ba7a33d575f4453dbf1b8861634a3f98 (table default.1_2 [id=44b956d4571840a780d714244fff640c])
I20260807 07:52:31.129113 6684 tablet_loader.cc:96] loaded metadata for tablet bcabb79073b040bfa91e1733e3d96128 (table default.0_4 [id=b198b5c57b4d4f89a279ae74df9fe9fc])
I20260807 07:52:31.129549 6684 tablet_loader.cc:96] loaded metadata for tablet de31825f9a984012abe2d537ea2b84f5 (table default.0_2 [id=fe68c09fd0234388b9d67f149d2bf73e])
I20260807 07:52:31.130023 6684 tablet_loader.cc:96] loaded metadata for tablet dea58cd0bc524d3e842a2feb335a6bfe (table default.1_4 [id=d44dab2abaf647b0a118ad2af02204d0])
I20260807 07:52:31.130425 6684 tablet_loader.cc:96] loaded metadata for tablet e25a88e31cf744e8b38bc2996babb16a (table default.0_5 [id=73efb1d25ad24e9bb7ad4fcd27db89e0])
I20260807 07:52:31.130836 6684 tablet_loader.cc:96] loaded metadata for tablet fda7602334414a979800d942a24cfb86 (table default.0_1 [id=f14c18ff0b174c4bb60578ae08177c6c])
I20260807 07:52:31.131225 6684 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:31.133013 6684 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:31.133117 6684 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:31.135435 6684 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:31.136561 6684 catalog_manager.cc:6081] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a: Loaded TSK: 0
I20260807 07:52:31.137454 6684 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:31.141883 6691 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:31.142004 6691 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: loading cluster ID for follower catalog manager: success
I20260807 07:52:31.147199 6691 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: acquiring CA information for follower catalog manager: success
I20260807 07:52:31.149931 6691 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:31.183223 6408 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53938:
name: "default.1_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:31.186182 6407 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53954:
name: "default.0_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:31.187058 6558 master_runner.cc:187] Error getting master registration for 127.2.169.251:34401: Network error: Client connection negotiation failed: client connection to 127.2.169.251:34401: connect: Connection refused (error 111),
I20260807 07:52:31.194095 6558 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:31.382195 6660 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:31.382550 6660 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:31.382694 6660 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:31.391590 6660 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:31.391674 6660 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:31.391754 6660 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:31.391819 6660 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:31.401918 6660 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.251:34401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.2.169.251
--webserver_port=43555
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:31.403514 6660 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:31.405438 6660 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:31.413807 6704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:31.414812 6705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:31.415555 6707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:31.416728 6660 server_base.cc:1061] running on GCE node
I20260807 07:52:31.417578 6660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:31.419116 6660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:31.420338 6660 hybrid_clock.cc:648] HybridClock initialized: now 1786089151420290 us; error 44 us; skew 500 ppm
I20260807 07:52:31.420789 6660 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:31.423363 6660 webserver.cc:533] Webserver started at http://127.2.169.251:43555/ using document root <none> and password file <none>
I20260807 07:52:31.424109 6660 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:31.424206 6660 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:31.430084 6660 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 07:52:31.433763 6715 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:31.435335 6660 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 07:52:31.435519 6660 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
uuid: "ea850e24aa3845a08ce29d1510612bb7"
format_stamp: "Formatted at 2026-08-07 07:52:21 on dist-test-slave-bw88"
I20260807 07:52:31.436264 6660 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:31.464038 6660 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:31.464967 6660 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:31.465277 6660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:31.488715 6660 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.251:34401
I20260807 07:52:31.488715 6778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.251:34401 every 8 connection(s)
I20260807 07:52:31.490653 6660 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-3/data/info.pb
I20260807 07:52:31.492201 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6660
I20260807 07:52:31.492996 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.250:34449
--webserver_interface=127.2.169.250
--webserver_port=42927
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:31.499126 6779 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:31.502907 6779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap starting.
I20260807 07:52:31.516708 6630 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [term 4 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 67. Preceding OpId from leader: term: 4 index: 68. (index mismatch)
I20260807 07:52:31.518049 6680 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 68, Last known committed idx: 67, Time since last communication: 0.000s
I20260807 07:52:31.529613 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.529884 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.542356 6779 log.cc:826] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:31.615360 6779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=67 overwritten=0 applied=67 ignored=0} inserts{seen=29 ignored=0} mutations{seen=46 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:31.616230 6779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: Bootstrap complete.
I20260807 07:52:31.626462 6779 raft_consensus.cc:359] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.627552 6779 raft_consensus.cc:740] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea850e24aa3845a08ce29d1510612bb7, State: Initialized, Role: FOLLOWER
I20260807 07:52:31.628461 6779 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:31.629617 6786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.629936 6786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.630745 6779 sys_catalog.cc:565] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:31.643934 6797 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:31.644112 6797 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: loading cluster ID for follower catalog manager: success
I20260807 07:52:31.651008 6797 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: acquiring CA information for follower catalog manager: success
I20260807 07:52:31.654196 6797 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:31.689949 6660 master_runner.cc:187] Error getting master registration for 127.2.169.250:34449: Network error: Client connection negotiation failed: client connection to 127.2.169.250:34449: connect: Connection refused (error 111),
I20260807 07:52:31.694135 6660 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260807 07:52:31.700870 6389 consensus_peers.cc:597] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a -> Peer 8a7e184f10af48e9a62e962cf40db93a (127.2.169.250:34449): Couldn't send request to peer 8a7e184f10af48e9a62e962cf40db93a. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.250:34449: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:31.706612 6523 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [term 4 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 67. Preceding OpId from leader: term: 4 index: 68. (index mismatch)
I20260807 07:52:31.707755 6680 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 68, Last known committed idx: 67, Time since last communication: 0.000s
I20260807 07:52:31.708438 6747 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:31.716254 6747 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [term 4 FOLLOWER]: Refusing update from remote peer ca5ccffd14db4e15be5ec8b65fc2502a: Log matching property violated. Preceding OpId in replica: term: 3 index: 67. Preceding OpId from leader: term: 4 index: 68. (index mismatch)
I20260807 07:52:31.717447 6680 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 68, Last known committed idx: 67, Time since last communication: 0.000s
I20260807 07:52:31.721879 6804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.722231 6804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.736171 6786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.736428 6786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.744400 6804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.744724 6804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a486fca3ed6e467daa9af0e316f2e075 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.754364 6811 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.754709 6811 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:31.755187 6680 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.755405 6680 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:31.755669 6680 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.755901 6680 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:31.759461 6786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.759698 6786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea850e24aa3845a08ce29d1510612bb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:31.759662 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:31.759915 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e50b67caa47fb95d0c1e1beefd731 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 07:52:31.940656 6782 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:31.940963 6782 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:31.941107 6782 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:31.949793 6782 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:31.949877 6782 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:31.949960 6782 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:31.950053 6782 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:31.960177 6782 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.250:34449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
--webserver_interface=127.2.169.250
--webserver_port=42927
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:31.961922 6782 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:31.963943 6782 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:31.972872 6821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:31.973116 6823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:31.973832 6825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:31.974484 6782 server_base.cc:1061] running on GCE node
I20260807 07:52:31.975247 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:31.976686 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:31.977917 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786089151977860 us; error 53 us; skew 500 ppm
I20260807 07:52:31.978353 6782 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:31.980710 6782 webserver.cc:533] Webserver started at http://127.2.169.250:42927/ using document root <none> and password file <none>
I20260807 07:52:31.981535 6782 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:31.981678 6782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:31.987253 6782 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 07:52:31.990329 6834 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:31.991814 6782 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 07:52:31.991991 6782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
uuid: "8a7e184f10af48e9a62e962cf40db93a"
format_stamp: "Formatted at 2026-08-07 07:52:26 on dist-test-slave-bw88"
I20260807 07:52:31.992723 6782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:32.017700 6782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:32.018638 6782 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:32.018977 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:32.048157 6782 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.250:34449
I20260807 07:52:32.048157 6894 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.250:34449 every 8 connection(s)
I20260807 07:52:32.049993 6782 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/master-4/data/info.pb
I20260807 07:52:32.057969 6895 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:32.060150 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6782
I20260807 07:52:32.061062 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:36591
--local_ip_for_outbound_sockets=127.2.169.193
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589
--webserver_port=34149
--webserver_interface=127.2.169.193
--builtin_ntp_servers=127.2.169.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449 with env {}
I20260807 07:52:32.062855 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap starting.
I20260807 07:52:32.099367 6895 log.cc:826] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:32.170985 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap replayed 1/1 log segments. Stats: ops{read=67 overwritten=0 applied=67 ignored=0} inserts{seen=29 ignored=0} mutations{seen=46 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.171762 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: Bootstrap complete.
I20260807 07:52:32.182675 6895 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:32.183689 6895 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7e184f10af48e9a62e962cf40db93a, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.184530 6895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 67, Last appended: 3.67, Last appended by leader: 67, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } }
I20260807 07:52:32.185642 6900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:32.186051 6900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:32.186878 6895 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:32.234375 6407 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54008:
name: "default.1_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:32.273857 6407 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54012:
name: "default.0_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:32.274636 6934 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 07:52:32.285386 6867 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:32.296614 6945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:32.297762 6945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:32.299135 6900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5ccffd14db4e15be5ec8b65fc2502a. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:32.299386 6900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 07:52:32.518388 6897 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:32.518761 6897 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:32.518908 6897 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:32.527540 6897 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:32.527756 6897 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:32.537832 6897 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:36591
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=34149
--tserver_master_addrs=127.2.169.254:45309,127.2.169.253:45589,127.2.169.252:39947,127.2.169.251:34401,127.2.169.250:34449
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:32.539707 6897 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:32.541718 6897 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:32.551458 6951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:32.551505 6952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:32.552289 6954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:32.553485 6897 server_base.cc:1061] running on GCE node
I20260807 07:52:32.554071 6897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:32.555027 6897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:32.556263 6897 hybrid_clock.cc:648] HybridClock initialized: now 1786089152556230 us; error 44 us; skew 500 ppm
I20260807 07:52:32.556586 6897 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:32.558833 6897 webserver.cc:533] Webserver started at http://127.2.169.193:34149/ using document root <none> and password file <none>
I20260807 07:52:32.559628 6897 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:32.559765 6897 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:32.566708 6897 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 07:52:32.571257 6961 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:32.572762 6897 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 07:52:32.572981 6897 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "e687b4c5396b431e8eeddf0065f474c1"
format_stamp: "Formatted at 2026-08-07 07:52:15 on dist-test-slave-bw88"
I20260807 07:52:32.573827 6897 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:32.609529 6897 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:32.610468 6897 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:32.610774 6897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:32.611785 6897 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:32.614336 6968 ts_tablet_manager.cc:548] Loading tablet metadata (0/13 complete)
I20260807 07:52:32.643775 6897 ts_tablet_manager.cc:585] Loaded tablet metadata (13 total tablets, 13 live tablets)
I20260807 07:52:32.643882 6897 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.030s user 0.002s sys 0.000s
I20260807 07:52:32.643992 6897 ts_tablet_manager.cc:600] Registering tablets (0/13 complete)
I20260807 07:52:32.647836 6968 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.671310 6968 log.cc:826] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:32.676543 6968 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.677197 6968 tablet_bootstrap.cc:492] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.678977 6968 ts_tablet_manager.cc:1403] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.020s
I20260807 07:52:32.680536 6897 ts_tablet_manager.cc:616] Registered 13 tablets
I20260807 07:52:32.680650 6897 ts_tablet_manager.cc:595] Time spent register tablets: real 0.037s user 0.032s sys 0.003s
I20260807 07:52:32.689636 6968 raft_consensus.cc:359] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.690956 6968 raft_consensus.cc:740] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.692052 6968 consensus_queue.cc:260] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.692344 6968 raft_consensus.cc:399] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.692518 6968 raft_consensus.cc:493] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.692727 6968 raft_consensus.cc:3060] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:32.693878 6811 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "ca5ccffd14db4e15be5ec8b65fc2502a" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 45309 } } peers { permanent_uuid: "a486fca3ed6e467daa9af0e316f2e075" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45589 } } peers { permanent_uuid: "279e50b67caa47fb95d0c1e1beefd731" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 39947 } attrs { promote: false } } peers { permanent_uuid: "ea850e24aa3845a08ce29d1510612bb7" member_type: VOTER last_known_addr { host: "127.2.169.251" port: 34401 } attrs { promote: false } } peers { permanent_uuid: "8a7e184f10af48e9a62e962cf40db93a" member_type: VOTER last_known_addr { host: "127.2.169.250" port: 34449 } attrs { promote: false } } }
I20260807 07:52:32.694222 6811 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5ccffd14db4e15be5ec8b65fc2502a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:32.699759 6968 raft_consensus.cc:515] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.700519 6968 leader_election.cc:304] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.704795 6987 raft_consensus.cc:2804] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:32.705670 6968 leader_election.cc:290] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:32.706664 6987 raft_consensus.cc:697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.707355 6987 consensus_queue.cc:237] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.714828 6968 ts_tablet_manager.cc:1434] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.036s user 0.025s sys 0.006s
I20260807 07:52:32.715466 6968 tablet_bootstrap.cc:492] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.738368 6968 tablet_bootstrap.cc:492] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.738976 6968 tablet_bootstrap.cc:492] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.740494 6968 ts_tablet_manager.cc:1403] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.015s
I20260807 07:52:32.742096 6968 raft_consensus.cc:359] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.742406 6968 raft_consensus.cc:740] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.742898 6968 consensus_queue.cc:260] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.743212 6968 raft_consensus.cc:399] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.743397 6968 raft_consensus.cc:493] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.743525 6968 raft_consensus.cc:3060] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:32.748456 6968 raft_consensus.cc:515] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.748845 6968 leader_election.cc:304] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.749539 6987 raft_consensus.cc:2804] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:32.749509 6968 leader_election.cc:290] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:32.749776 6987 raft_consensus.cc:697] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.750012 6987 consensus_queue.cc:237] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.750222 6968 ts_tablet_manager.cc:1434] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260807 07:52:32.750650 6968 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.764410 6897 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:36591
I20260807 07:52:32.764449 7111 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:36591 every 8 connection(s)
I20260807 07:52:32.766638 6897 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.AutoAddMasterTest.TestAddWithOnGoingDdl.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:32.773418 6968 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.774082 6968 tablet_bootstrap.cc:492] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.775825 6968 ts_tablet_manager.cc:1403] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.012s
I20260807 07:52:32.776343 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 6897
I20260807 07:52:32.776958 6968 raft_consensus.cc:359] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.777194 6968 raft_consensus.cc:740] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.778496 6968 consensus_queue.cc:260] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.778733 6968 raft_consensus.cc:399] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.778837 6968 raft_consensus.cc:493] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.778931 6968 raft_consensus.cc:3060] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:32.782166 6968 raft_consensus.cc:515] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.782387 6968 leader_election.cc:304] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.782578 6968 leader_election.cc:290] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260807 07:52:32.782681 6987 raft_consensus.cc:2804] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:32.782892 6987 raft_consensus.cc:697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [term 4 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.783069 6968 ts_tablet_manager.cc:1434] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 07:52:32.783200 6987 consensus_queue.cc:237] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.783320 6968 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.814385 6968 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.815066 6968 tablet_bootstrap.cc:492] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.816337 7114 heartbeater.cc:344] Connected to a master server at 127.2.169.251:34401
I20260807 07:52:32.816815 7114 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:32.816824 6968 ts_tablet_manager.cc:1403] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.005s
I20260807 07:52:32.818169 7114 heartbeater.cc:507] Master 127.2.169.251:34401 requested a full tablet report, sending...
I20260807 07:52:32.818256 6968 raft_consensus.cc:359] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.818493 6968 raft_consensus.cc:740] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.818817 6968 consensus_queue.cc:260] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.819036 6968 raft_consensus.cc:399] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.819165 6968 raft_consensus.cc:493] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.819296 6968 raft_consensus.cc:3060] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:32.823117 7113 heartbeater.cc:344] Connected to a master server at 127.2.169.250:34449
I20260807 07:52:32.823292 7113 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:32.823630 6968 raft_consensus.cc:515] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.823884 7113 heartbeater.cc:507] Master 127.2.169.250:34449 requested a full tablet report, sending...
I20260807 07:52:32.826403 7117 heartbeater.cc:344] Connected to a master server at 127.2.169.254:45309
I20260807 07:52:32.826551 7117 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:32.827147 7117 heartbeater.cc:507] Master 127.2.169.254:45309 requested a full tablet report, sending...
I20260807 07:52:32.827622 6968 leader_election.cc:304] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.828054 6987 raft_consensus.cc:2804] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:32.828384 6987 raft_consensus.cc:697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [term 4 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.828694 6987 consensus_queue.cc:237] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.829511 6968 leader_election.cc:290] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260807 07:52:32.830432 6968 ts_tablet_manager.cc:1434] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260807 07:52:32.830776 6968 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.831782 6407 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:32.835810 6734 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:32.837561 6854 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:32.849021 6407 catalog_manager.cc:5697] T fda7602334414a979800d942a24cfb86 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:32.853497 6407 catalog_manager.cc:5697] T dea58cd0bc524d3e842a2feb335a6bfe P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:32.853979 6407 catalog_manager.cc:5697] T 6e479075a88046939401633792a640aa P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:32.862557 6968 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.863257 6968 tablet_bootstrap.cc:492] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.865095 6968 ts_tablet_manager.cc:1403] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.034s user 0.013s sys 0.016s
I20260807 07:52:32.866333 6968 raft_consensus.cc:359] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.866585 6968 raft_consensus.cc:740] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.867092 6968 consensus_queue.cc:260] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.867303 6968 raft_consensus.cc:399] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.867439 6968 raft_consensus.cc:493] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.867583 6968 raft_consensus.cc:3060] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:32.871737 6968 raft_consensus.cc:515] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.872571 6968 leader_election.cc:304] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.872800 6968 leader_election.cc:290] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260807 07:52:32.872844 6987 raft_consensus.cc:2804] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:32.873114 6987 raft_consensus.cc:697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.873230 6968 ts_tablet_manager.cc:1434] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.008s user 0.006s sys 0.004s
I20260807 07:52:32.873543 6968 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.873417 6987 consensus_queue.cc:237] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.880556 7115 heartbeater.cc:344] Connected to a master server at 127.2.169.252:39947
I20260807 07:52:32.880728 7115 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:32.881312 7115 heartbeater.cc:507] Master 127.2.169.252:39947 requested a full tablet report, sending...
I20260807 07:52:32.902515 6968 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.903050 6968 tablet_bootstrap.cc:492] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.904410 6968 ts_tablet_manager.cc:1403] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.016s
I20260807 07:52:32.904767 7118 heartbeater.cc:344] Connected to a master server at 127.2.169.253:45589
I20260807 07:52:32.904909 7118 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:32.905467 7118 heartbeater.cc:507] Master 127.2.169.253:45589 requested a full tablet report, sending...
I20260807 07:52:32.906599 6968 raft_consensus.cc:359] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.906786 6968 raft_consensus.cc:740] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.907066 6968 consensus_queue.cc:260] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.907219 6968 raft_consensus.cc:399] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.907313 6968 raft_consensus.cc:493] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.907397 6968 raft_consensus.cc:3060] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:32.910352 6968 raft_consensus.cc:515] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.911171 6968 leader_election.cc:304] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.911393 6968 leader_election.cc:290] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:32.911432 6987 raft_consensus.cc:2804] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:32.911613 6987 raft_consensus.cc:697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.911859 6987 consensus_queue.cc:237] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.918622 6968 ts_tablet_manager.cc:1434] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.014s user 0.005s sys 0.010s
I20260807 07:52:32.918805 6620 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:32.918988 6968 tablet_bootstrap.cc:492] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.931370 6507 ts_manager.cc:194] Registered new tserver with Master: e687b4c5396b431e8eeddf0065f474c1 (127.2.169.193:36591)
I20260807 07:52:32.950145 6968 tablet_bootstrap.cc:492] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:32.950721 6968 tablet_bootstrap.cc:492] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:32.951774 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:32.952240 6968 ts_tablet_manager.cc:1403] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.013s
I20260807 07:52:32.953425 6968 raft_consensus.cc:359] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.953704 6968 raft_consensus.cc:740] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:32.954077 6968 consensus_queue.cc:260] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.954277 6968 raft_consensus.cc:399] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:32.954380 6968 raft_consensus.cc:493] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:32.954504 6968 raft_consensus.cc:3060] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:32.958640 6968 raft_consensus.cc:515] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.958930 6968 leader_election.cc:304] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:32.961033 6987 raft_consensus.cc:2804] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:32.962141 6987 raft_consensus.cc:697] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:32.962409 6987 consensus_queue.cc:237] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:32.964653 6968 leader_election.cc:290] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:32.965504 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:32.965672 6968 ts_tablet_manager.cc:1434] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20260807 07:52:32.966022 6968 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:32.989492 6407 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:55057
I20260807 07:52:32.996263 7117 heartbeater.cc:499] Master 127.2.169.254:45309 was elected leader, sending a full tablet report...
I20260807 07:52:33.004734 6407 catalog_manager.cc:5697] T e25a88e31cf744e8b38bc2996babb16a P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.005306 6407 catalog_manager.cc:5697] T 23de065eeead41b8aaec4866731369f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.006060 6407 catalog_manager.cc:5697] T 7c30a1b53ff54e9a8515149bd3d5d84f P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.006759 6407 catalog_manager.cc:5697] T 5036daa1e5ed493db4a152f70bae715b P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.011075 6968 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.011895 6968 tablet_bootstrap.cc:492] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.013310 6968 ts_tablet_manager.cc:1403] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.047s user 0.018s sys 0.022s
I20260807 07:52:33.014268 6968 raft_consensus.cc:359] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.014430 6968 raft_consensus.cc:740] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.014889 6968 consensus_queue.cc:260] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.015122 6968 raft_consensus.cc:399] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.015231 6968 raft_consensus.cc:493] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.015326 6968 raft_consensus.cc:3060] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:33.018311 6968 raft_consensus.cc:515] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.018559 6968 leader_election.cc:304] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.018826 6987 raft_consensus.cc:2804] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:33.019003 6987 raft_consensus.cc:697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [term 4 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.019223 6987 consensus_queue.cc:237] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.025255 6968 leader_election.cc:290] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260807 07:52:33.025758 6968 ts_tablet_manager.cc:1434] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.012s user 0.002s sys 0.003s
I20260807 07:52:33.026075 6968 tablet_bootstrap.cc:492] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:33.040200 6407 catalog_manager.cc:5697] T ba7a33d575f4453dbf1b8861634a3f98 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.059142 6968 tablet_bootstrap.cc:492] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.059847 6968 tablet_bootstrap.cc:492] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.061021 6968 ts_tablet_manager.cc:1403] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.026s
I20260807 07:52:33.061971 6968 raft_consensus.cc:359] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.062165 6968 raft_consensus.cc:740] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.062450 6968 consensus_queue.cc:260] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.062692 6968 raft_consensus.cc:399] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.062821 6968 raft_consensus.cc:493] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.062937 6968 raft_consensus.cc:3060] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:33.064992 6968 raft_consensus.cc:515] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.065225 6968 leader_election.cc:304] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.065519 6968 leader_election.cc:290] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:33.065649 7142 raft_consensus.cc:2804] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:33.065887 7142 raft_consensus.cc:697] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.066051 6968 ts_tablet_manager.cc:1434] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 07:52:33.066418 6968 tablet_bootstrap.cc:492] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:33.066411 7142 consensus_queue.cc:237] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.075328 6407 catalog_manager.cc:5697] T 000ff4da6ee741c68edcce5bebf25460 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.100831 6968 tablet_bootstrap.cc:492] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.101646 6968 tablet_bootstrap.cc:492] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.102695 6968 ts_tablet_manager.cc:1403] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.024s
I20260807 07:52:33.103338 6968 raft_consensus.cc:359] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.103482 6968 raft_consensus.cc:740] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.103739 6968 consensus_queue.cc:260] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.103866 6968 raft_consensus.cc:399] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.103945 6968 raft_consensus.cc:493] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.104012 6968 raft_consensus.cc:3060] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:33.105926 6968 raft_consensus.cc:515] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.106112 6968 leader_election.cc:304] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.106323 6968 leader_election.cc:290] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:33.106483 7142 raft_consensus.cc:2804] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:33.107010 7142 raft_consensus.cc:697] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [term 2 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.107091 6968 ts_tablet_manager.cc:1434] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260807 07:52:33.107312 7142 consensus_queue.cc:237] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.107795 6968 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:33.115962 6407 catalog_manager.cc:5697] T 3ce0b4ae485c453aaab7a4c0ec4a75e2 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.136466 6968 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.137058 6968 tablet_bootstrap.cc:492] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.138198 6968 ts_tablet_manager.cc:1403] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.022s
I20260807 07:52:33.138839 6968 raft_consensus.cc:359] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.139008 6968 raft_consensus.cc:740] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.139287 6968 consensus_queue.cc:260] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.139458 6968 raft_consensus.cc:399] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.139554 6968 raft_consensus.cc:493] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.139667 6968 raft_consensus.cc:3060] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:33.141721 6968 raft_consensus.cc:515] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.141947 6968 leader_election.cc:304] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.142153 6968 leader_election.cc:290] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 4 election: Requested vote from peers
I20260807 07:52:33.142261 7142 raft_consensus.cc:2804] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:33.142515 7142 raft_consensus.cc:697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [term 4 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.142732 6968 ts_tablet_manager.cc:1434] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260807 07:52:33.142765 7142 consensus_queue.cc:237] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.143064 6968 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:33.149497 6407 catalog_manager.cc:5697] T de31825f9a984012abe2d537ea2b84f5 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.170722 6968 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.171396 6968 tablet_bootstrap.cc:492] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.172425 6968 ts_tablet_manager.cc:1403] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.018s
I20260807 07:52:33.173048 6968 raft_consensus.cc:359] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.173180 6968 raft_consensus.cc:740] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.173544 6968 consensus_queue.cc:260] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.173671 6968 raft_consensus.cc:399] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.173794 6968 raft_consensus.cc:493] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.173915 6968 raft_consensus.cc:3060] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:33.175836 6968 raft_consensus.cc:515] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.176050 6968 leader_election.cc:304] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.176280 6968 leader_election.cc:290] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:33.176589 7142 raft_consensus.cc:2804] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:33.176774 6968 ts_tablet_manager.cc:1434] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.004s user 0.000s sys 0.007s
I20260807 07:52:33.176829 7142 raft_consensus.cc:697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.177142 7142 consensus_queue.cc:237] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.177146 6968 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap starting.
I20260807 07:52:33.183315 6407 catalog_manager.cc:5697] T 1298cbdc19e640b4bbbeeaf1c34d73bf P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.204692 6968 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:33.205297 6968 tablet_bootstrap.cc:492] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Bootstrap complete.
I20260807 07:52:33.206390 6968 ts_tablet_manager.cc:1403] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.025s
I20260807 07:52:33.207134 6968 raft_consensus.cc:359] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.207279 6968 raft_consensus.cc:740] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Initialized, Role: FOLLOWER
I20260807 07:52:33.207489 6968 consensus_queue.cc:260] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.207603 6968 raft_consensus.cc:399] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:33.207669 6968 raft_consensus.cc:493] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:33.207732 6968 raft_consensus.cc:3060] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:33.209538 6968 raft_consensus.cc:515] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.209703 6968 leader_election.cc:304] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e687b4c5396b431e8eeddf0065f474c1; no voters:
I20260807 07:52:33.209897 6968 leader_election.cc:290] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:52:33.210116 7142 raft_consensus.cc:2804] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:33.210343 7142 raft_consensus.cc:697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [term 3 LEADER]: Becoming Leader. State: Replica: e687b4c5396b431e8eeddf0065f474c1, State: Running, Role: LEADER
I20260807 07:52:33.210506 6968 ts_tablet_manager.cc:1434] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 07:52:33.210688 7142 consensus_queue.cc:237] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } }
I20260807 07:52:33.213958 6407 catalog_manager.cc:5697] T bcabb79073b040bfa91e1733e3d96128 P e687b4c5396b431e8eeddf0065f474c1 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "e687b4c5396b431e8eeddf0065f474c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e687b4c5396b431e8eeddf0065f474c1" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 36591 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:33.277405 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6897
I20260807 07:52:33.281256 6934 catalog_manager.cc:1295] Loaded cluster ID: 5e85e88a2d2542d4a11d05242de0e553
I20260807 07:52:33.281411 6934 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: loading cluster ID for follower catalog manager: success
I20260807 07:52:33.285102 6934 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: acquiring CA information for follower catalog manager: success
I20260807 07:52:33.288022 6934 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8a7e184f10af48e9a62e962cf40db93a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:33.289172 7107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:33.424005 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6897
I20260807 07:52:33.464479 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6367
I20260807 07:52:33.466038 6441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:33.596534 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6367
I20260807 07:52:33.640013 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6452
I20260807 07:52:33.641443 6549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:33.755021 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6452
I20260807 07:52:33.795691 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6558
I20260807 07:52:33.797271 6653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:33.928071 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6558
I20260807 07:52:33.954734 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6660
I20260807 07:52:33.956195 6773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:34.062757 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6660
I20260807 07:52:34.090658 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 6782
I20260807 07:52:34.091881 6889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:34.199016 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 6782
2026-08-07T07:52:34Z chronyd exiting
[ OK ] AutoAddMasterTest.TestAddWithOnGoingDdl (20279 ms)
[----------] 1 test from AutoAddMasterTest (20279 ms total)
[----------] 1 test from ParameterizedAddMasterTest
[ RUN ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/3
Loading random data
Initializing database '/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 07:52:34 dist-test-slave-bw88 krb5kdc[7190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 07:52:34 dist-test-slave-bw88 krb5kdc[7190](info): set up 2 sockets
Aug 07 07:52:34 dist-test-slave-bw88 krb5kdc[7190](info): commencing operation
krb5kdc: starting...
W20260807 07:52:36.441308 2727 mini_kdc.cc:121] Time spent starting KDC: real 2.182s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Aug 07 07:52:36 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T07:52:36Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.169.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:36.741659 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:33341
--webserver_interface=127.2.169.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:33341,127.2.169.253:44995
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:37.092799 7218 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:37.093158 7218 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:37.093328 7218 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:37.101830 7218 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:37.101979 7218 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:37.102094 7218 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:37.102177 7218 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:37.112771 7218 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:33341,127.2.169.253:44995
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:33341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:37.113152 7218 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:33341,127.2.169.253:44995'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:37.114620 7218 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:37.116050 7218 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:37.124886 7231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:37.125198 7230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:37.126185 7234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:37.126332 7218 server_base.cc:1061] running on GCE node
I20260807 07:52:37.127179 7218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:37.128527 7218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:37.129745 7218 hybrid_clock.cc:648] HybridClock initialized: now 1786089157129690 us; error 23 us; skew 500 ppm
I20260807 07:52:37.130040 7218 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:37.133589 7218 init.cc:377] Logged in from keytab as kudu/127.2.169.254@KRBTEST.COM (short username kudu)
I20260807 07:52:37.135255 7218 webserver.cc:533] Webserver started at http://127.2.169.254:34229/ using document root <none> and password file <none>
I20260807 07:52:37.135998 7218 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:37.136121 7218 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:37.136535 7218 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:37.139025 7218 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "872659a463ac417d9526c07b6f443c21"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.139835 7218 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "872659a463ac417d9526c07b6f443c21"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.145565 7218 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 07:52:37.149003 7243 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:37.150480 7218 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:37.150676 7218 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "872659a463ac417d9526c07b6f443c21"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.150904 7218 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:37.166746 7218 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:37.175098 7218 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:37.175375 7218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:37.196316 7218 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:33341
I20260807 07:52:37.196338 7301 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:33341 every 8 connection(s)
I20260807 07:52:37.197880 7218 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:37.202150 7302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:37.203364 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 7218
I20260807 07:52:37.203665 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:37.211105 7302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.169.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.253@KRBTEST.COM" created.
I20260807 07:52:37.234591 7302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:52:37.235875 7246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
W20260807 07:52:37.237682 7302 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:44995: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.253@KRBTEST.COM" created.
I20260807 07:52:37.287009 7302 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } attempt: 1
W20260807 07:52:37.288992 7302 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:44995: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:37.304354 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:44995
--webserver_interface=127.2.169.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:33341,127.2.169.253:44995
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 07:52:37.357267 7302 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } attempt: 2
W20260807 07:52:37.359634 7302 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:44995: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
I20260807 07:52:37.450959 7302 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } attempt: 3
W20260807 07:52:37.453404 7302 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:44995: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
I20260807 07:52:37.596714 7302 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } attempt: 4
W20260807 07:52:37.599085 7302 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:44995: Network error: Client connection negotiation failed: client connection to 127.2.169.253:44995: connect: Connection refused (error 111)
W20260807 07:52:37.659965 7313 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:37.660271 7313 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:37.660380 7313 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:37.669174 7313 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:37.669277 7313 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:37.669353 7313 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:37.669435 7313 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:37.680013 7313 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:33341,127.2.169.253:44995
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:37.680402 7313 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:33341,127.2.169.253:44995'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:37.681891 7313 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:37.683350 7313 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:37.691686 7320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:37.691754 7318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:37.693537 7322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:37.694015 7313 server_base.cc:1061] running on GCE node
I20260807 07:52:37.694715 7313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:37.696063 7313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:37.697289 7313 hybrid_clock.cc:648] HybridClock initialized: now 1786089157697208 us; error 55 us; skew 500 ppm
I20260807 07:52:37.697654 7313 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:37.700963 7313 init.cc:377] Logged in from keytab as kudu/127.2.169.253@KRBTEST.COM (short username kudu)
I20260807 07:52:37.702771 7313 webserver.cc:533] Webserver started at http://127.2.169.253:34929/ using document root <none> and password file <none>
I20260807 07:52:37.703554 7313 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:37.703697 7313 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:37.704103 7313 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:37.706619 7313 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "407070fcf89447f39beccb665abac23a"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.707492 7313 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "407070fcf89447f39beccb665abac23a"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.712941 7313 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 07:52:37.716500 7331 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:37.717954 7313 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:37.718135 7313 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "407070fcf89447f39beccb665abac23a"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:37.718323 7313 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:37.763422 7313 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:37.767553 7313 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:37.767791 7313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:37.789090 7313 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:44995
I20260807 07:52:37.789124 7402 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:44995 every 8 connection(s)
I20260807 07:52:37.790693 7313 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:37.795186 7403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:37.796484 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 7313
I20260807 07:52:37.796715 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:52:37.802789 7403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:37.824870 7403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:37.843686 7403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Bootstrap starting.
I20260807 07:52:37.847074 7403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:37.848357 7403 log.cc:826] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:37.850840 7403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: No bootstrap required, opened a new log
I20260807 07:52:37.856564 7403 raft_consensus.cc:359] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.856858 7403 raft_consensus.cc:385] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:37.856961 7403 raft_consensus.cc:740] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407070fcf89447f39beccb665abac23a, State: Initialized, Role: FOLLOWER
I20260807 07:52:37.857643 7403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.858547 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:37.858811 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:37.859545 7403 sys_catalog.cc:565] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:37.868952 7424 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:37.869107 7424 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:37.898345 7302 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } attempt: 5
Aug 07 07:52:37 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:37.917449 7302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Bootstrap starting.
I20260807 07:52:37.920950 7302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:37.922227 7302 log.cc:826] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:37.925535 7302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: No bootstrap required, opened a new log
I20260807 07:52:37.931506 7302 raft_consensus.cc:359] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.931926 7302 raft_consensus.cc:385] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:37.932075 7302 raft_consensus.cc:740] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872659a463ac417d9526c07b6f443c21, State: Initialized, Role: FOLLOWER
I20260807 07:52:37.932802 7302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.933879 7428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:37.934216 7428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:37.935084 7302 sys_catalog.cc:565] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:37.945891 7443 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:37.946066 7443 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:37.985538 7412 raft_consensus.cc:493] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:37.985917 7412 raft_consensus.cc:515] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.987958 7412 leader_election.cc:290] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872659a463ac417d9526c07b6f443c21 (127.2.169.254:33341)
I20260807 07:52:37.989058 7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872659a463ac417d9526c07b6f443c21" is_pre_election: true
I20260807 07:52:37.989674 7271 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407070fcf89447f39beccb665abac23a in term 0.
I20260807 07:52:37.990459 7338 leader_election.cc:304] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, 872659a463ac417d9526c07b6f443c21; no voters:
I20260807 07:52:37.991140 7412 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 07:52:37.991384 7412 raft_consensus.cc:493] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:37.991551 7412 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:37.995127 7412 raft_consensus.cc:515] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:37.995786 7412 leader_election.cc:290] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 1 election: Requested vote from peers 872659a463ac417d9526c07b6f443c21 (127.2.169.254:33341)
I20260807 07:52:37.996378 7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872659a463ac417d9526c07b6f443c21"
I20260807 07:52:37.996639 7271 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:38.000262 7271 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407070fcf89447f39beccb665abac23a in term 1.
I20260807 07:52:38.000798 7338 leader_election.cc:304] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, 872659a463ac417d9526c07b6f443c21; no voters:
I20260807 07:52:38.001317 7412 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:38.001915 7412 raft_consensus.cc:697] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 LEADER]: Becoming Leader. State: Replica: 407070fcf89447f39beccb665abac23a, State: Running, Role: LEADER
I20260807 07:52:38.002573 7412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } }
I20260807 07:52:38.006752 7445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407070fcf89447f39beccb665abac23a. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:38.007099 7445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:38.008185 7447 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:38.010746 7447 catalog_manager.cc:1520] Initializing Kudu cluster ID...
W20260807 07:52:38.019876 7340 tablet.cc:2406] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:38.019927 7271 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Refusing update from remote peer 407070fcf89447f39beccb665abac23a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:52:38.021116 7445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 07:52:38.030328 7428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407070fcf89447f39beccb665abac23a. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:38.030580 7428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:38.032759 7448 mvcc.cc:204] Tried to move back new op lower bound from 7315821191237820416 to 7315821191183945728. Current Snapshot: MvccSnapshot[applied={T|T < 7315821191237820416}]
I20260807 07:52:38.037885 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:38.038323 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:38.040673 7447 catalog_manager.cc:1383] Generated new cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:38.040802 7447 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:38.044348 7428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } }
I20260807 07:52:38.044535 7428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:38.073297 7447 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:52:38.074777 7447 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:38.089097 7447 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Generated new TSK 0
I20260807 07:52:38.090092 7447 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.169.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:38.217494 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:33341,127.2.169.253:44995
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:38.595674 7458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:38.596061 7458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:38.596236 7458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:38.605080 7458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:38.605383 7458 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:38.617255 7458 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.2.169.254:33341,127.2.169.253:44995
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:38.619091 7458 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:38.620700 7458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:38.631644 7464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:38.632212 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:38.632754 7463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:38.633599 7458 server_base.cc:1061] running on GCE node
I20260807 07:52:38.634263 7458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:38.635351 7458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:38.636619 7458 hybrid_clock.cc:648] HybridClock initialized: now 1786089158636563 us; error 43 us; skew 500 ppm
I20260807 07:52:38.636982 7458 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:38 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089158, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:38.640700 7458 init.cc:377] Logged in from keytab as kudu/127.2.169.193@KRBTEST.COM (short username kudu)
I20260807 07:52:38.642439 7458 webserver.cc:533] Webserver started at http://127.2.169.193:45663/ using document root <none> and password file <none>
I20260807 07:52:38.643177 7458 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:38.643273 7458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:38.643600 7458 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:38.646173 7458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "8983fadcb93943dc9c97956e233f8787"
format_stamp: "Formatted at 2026-08-07 07:52:38 on dist-test-slave-bw88"
I20260807 07:52:38.646996 7458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "8983fadcb93943dc9c97956e233f8787"
format_stamp: "Formatted at 2026-08-07 07:52:38 on dist-test-slave-bw88"
I20260807 07:52:38.653220 7458 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 07:52:38.656903 7481 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:38.658370 7458 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:38.658565 7458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "8983fadcb93943dc9c97956e233f8787"
format_stamp: "Formatted at 2026-08-07 07:52:38 on dist-test-slave-bw88"
I20260807 07:52:38.658768 7458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:38.674942 7458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:38.678954 7458 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:38.679288 7458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:38.680351 7458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:38.682118 7458 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:52:38.682242 7458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:38.682421 7458 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:52:38.682508 7458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:38.719484 7458 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:43449
I20260807 07:52:38.719529 7621 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:43449 every 8 connection(s)
I20260807 07:52:38.721063 7458 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:38.726539 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 7458
I20260807 07:52:38.726818 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
Aug 07 07:52:38 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089158, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:52:38 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089158, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:38.762683 7622 heartbeater.cc:344] Connected to a master server at 127.2.169.253:44995
I20260807 07:52:38.763110 7622 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:38.764304 7622 heartbeater.cc:507] Master 127.2.169.253:44995 requested a full tablet report, sending...
I20260807 07:52:38.765120 7623 heartbeater.cc:344] Connected to a master server at 127.2.169.254:33341
I20260807 07:52:38.765266 7623 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:38.765738 7623 heartbeater.cc:507] Master 127.2.169.254:33341 requested a full tablet report, sending...
I20260807 07:52:38.766829 7353 ts_manager.cc:194] Registered new tserver with Master: 8983fadcb93943dc9c97956e233f8787 (127.2.169.193:43449)
I20260807 07:52:38.767431 7261 ts_manager.cc:194] Registered new tserver with Master: 8983fadcb93943dc9c97956e233f8787 (127.2.169.193:43449)
I20260807 07:52:38.768700 7353 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.169.193@KRBTEST.COM'} at 127.2.169.193:50577
I20260807 07:52:38.779016 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:38.784582 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:38.817215 7353 catalog_manager.cc:2283] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39868:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:38.848910 7539 tablet_service.cc:1511] Processing CreateTablet for tablet b217bdb88a2b419692eb9b29e3c6c799 (DEFAULT_TABLE table=first_table [id=8aff27aa969f476aadecf95fcf0f4c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:38.850374 7539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b217bdb88a2b419692eb9b29e3c6c799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:38.860177 7645 tablet_bootstrap.cc:492] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:38.863552 7645 tablet_bootstrap.cc:654] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:38.864637 7645 log.cc:826] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:38.867239 7645 tablet_bootstrap.cc:492] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: No bootstrap required, opened a new log
I20260807 07:52:38.867491 7645 ts_tablet_manager.cc:1403] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260807 07:52:38.872476 7645 raft_consensus.cc:359] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:38.872743 7645 raft_consensus.cc:385] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:38.872813 7645 raft_consensus.cc:740] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:38.873396 7645 consensus_queue.cc:260] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:38.873626 7645 raft_consensus.cc:399] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:38.873721 7645 raft_consensus.cc:493] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:38.873834 7645 raft_consensus.cc:3060] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:38.875753 7645 raft_consensus.cc:515] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:38.876143 7645 leader_election.cc:304] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:38.876708 7645 leader_election.cc:290] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:38.876885 7648 raft_consensus.cc:2804] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:38.877943 7622 heartbeater.cc:499] Master 127.2.169.253:44995 was elected leader, sending a full tablet report...
I20260807 07:52:38.878593 7648 raft_consensus.cc:697] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:38.878649 7645 ts_tablet_manager.cc:1434] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260807 07:52:38.879117 7648 consensus_queue.cc:237] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:38.885138 7353 catalog_manager.cc:5697] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8983fadcb93943dc9c97956e233f8787 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } health_report { overall_health: HEALTHY } } }
WARNING: no policy specified for kudu/127.2.169.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.252@KRBTEST.COM" created.
I20260807 07:52:38.948134 7443 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:38.948256 7443 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: loading cluster ID for follower catalog manager: success
I20260807 07:52:38.950919 7443 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: acquiring CA information for follower catalog manager: success
I20260807 07:52:38.952768 7443 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Aug 07 07:52:39 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089159, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 07:52:39 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089159, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:52:39 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089159, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
Aug 07 07:52:39 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089159, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for kudu/127.2.169.252@KRBTEST.COM
Aug 07 07:52:39 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.252@KRBTEST.COM
Aug 07 07:52:40 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089157, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for kudu/127.2.169.252@KRBTEST.COM
I20260807 07:52:40.079486 7353 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.2.169.252:33687: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c"
member_type: NON_VOTER
last_known_addr {
host: "127.2.169.252"
port: 33687
}
attrs {
promote: true
}
}
dest_uuid: "407070fcf89447f39beccb665abac23a"
cas_config_opid_index: -1
I20260807 07:52:40.080668 7353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } }
I20260807 07:52:40.083493 7271 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Refusing update from remote peer 407070fcf89447f39beccb665abac23a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 07:52:40.084301 7637 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 07:52:40.090574 7637 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 07:52:40.093446 7271 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.098306 7785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.098644 7785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:40.092692 7784 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.103511 7784 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.2.169.252:33687
I20260807 07:52:40.106297 7637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.106527 7637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:40.112396 7780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.112763 7780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:40.156936 7784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: NON_VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: true } } }
I20260807 07:52:40.157253 7784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:40.165910 7637 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: attempting to promote NON_VOTER c275a4ed29d74f89a2fd7fb95861867c to VOTER
I20260807 07:52:40.166916 7637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:40.169040 7271 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Refusing update from remote peer 407070fcf89447f39beccb665abac23a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 07:52:40.169688 7784 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 07:52:40.170017 7780 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 07:52:40.172510 7637 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.173296 7271 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.176752 7780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.176963 7780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:40.177248 7785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.177470 7785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:40.177977 7780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.178021 7784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:40.178224 7780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:40.178247 7784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for kudu/127.2.169.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
W20260807 07:52:40.660732 7332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.252:33687: connect: Connection refused (error 111)
W20260807 07:52:40.662741 7332 consensus_peers.cc:597] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a -> Peer c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687): Couldn't send request to peer c275a4ed29d74f89a2fd7fb95861867c. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.252:33687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:41.254477 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:41.254971 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
Aug 07 07:52:41 dist-test-slave-bw88 krb5kdc[7190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089161, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 07:52:41.663954 7353 master_service.cc:297] Master 127.2.169.252:33687 already present
WARNING: no policy specified for kudu/127.2.169.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:42.389659 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:42.390025 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
W20260807 07:52:42.402616 7332 consensus_peers.cc:597] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a -> Peer c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687): Couldn't send request to peer c275a4ed29d74f89a2fd7fb95861867c. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.252:33687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:42.759347 2727 dynamic_multi_master-test.cc:523] Add master stderr: W20260807 07:52:42.612874 8008 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:42.613183 8008 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:42.613279 8008 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:42.621632 8008 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:42.621738 8008 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:42.621802 8008 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:42.621853 8008 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
Illegal state: Failed bringing up new master. See Kudu Master log for details.: Master 127.2.169.254:33341 already running
I20260807 07:52:42.762919 2727 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:42.763049 2727 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:42.765859 2727 dynamic_multi_master-test.cc:614] Shutting down the old cluster
I20260807 07:52:42.765988 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 7458
Aug 07 07:52:42 dist-test-slave-bw88 krb5kdc[7190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.193@KRBTEST.COM
I20260807 07:52:42.781906 7617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:42.880635 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 7458
I20260807 07:52:42.908989 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 7218
I20260807 07:52:42.910346 7293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:42.968065 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:42.968387 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:43.030936 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 7218
I20260807 07:52:43.063861 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 7313
I20260807 07:52:43.065057 7394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:43.192487 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 7313
2026-08-07T07:52:43Z chronyd exiting
I20260807 07:52:43.238423 2727 dynamic_multi_master-test.cc:617] Bringing up the migrated cluster
Aug 07 07:52:43 dist-test-slave-bw88 krb5kdc[8044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 07:52:43 dist-test-slave-bw88 krb5kdc[8044](info): set up 2 sockets
Aug 07 07:52:43 dist-test-slave-bw88 krb5kdc[8044](info): commencing operation
krb5kdc: starting...
W20260807 07:52:45.474412 2727 mini_kdc.cc:121] Time spent starting KDC: real 2.233s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Aug 07 07:52:45 dist-test-slave-bw88 krb5kdc[8044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T07:52:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.169.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.254 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.254 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.254 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.254 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:45.732316 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:33341
--webserver_interface=127.2.169.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:46.077016 8063 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:46.077382 8063 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:46.077562 8063 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:46.085541 8063 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:46.085641 8063 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:46.085776 8063 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:46.085850 8063 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:46.095964 8063 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:33341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:46.097558 8063 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:46.098940 8063 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:46.107812 8069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:46.107885 8071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:46.108538 8073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:46.109218 8063 server_base.cc:1061] running on GCE node
I20260807 07:52:46.110015 8063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:46.111413 8063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:46.112625 8063 hybrid_clock.cc:648] HybridClock initialized: now 1786089166112551 us; error 58 us; skew 500 ppm
I20260807 07:52:46.112938 8063 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:46 dist-test-slave-bw88 krb5kdc[8044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:46.116434 8063 init.cc:377] Logged in from keytab as kudu/127.2.169.254@KRBTEST.COM (short username kudu)
I20260807 07:52:46.118196 8063 webserver.cc:533] Webserver started at http://127.2.169.254:34185/ using document root <none> and password file <none>
I20260807 07:52:46.119011 8063 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:46.119179 8063 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:46.124879 8063 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 07:52:46.127708 8082 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:46.129017 8063 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 07:52:46.129202 8063 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "872659a463ac417d9526c07b6f443c21"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:46.130040 8063 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:46.148233 8063 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:46.152377 8063 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:46.152722 8063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:46.174412 8063 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:33341
I20260807 07:52:46.174431 8147 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:33341 every 8 connection(s)
I20260807 07:52:46.176141 8063 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:46.183265 8148 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:46.183460 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8063
I20260807 07:52:46.183710 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:46.186795 8148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Bootstrap starting.
I20260807 07:52:46.204428 8148 log.cc:826] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.2.169.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260807 07:52:46.216315 8148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:46.216897 8148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Bootstrap complete.
I20260807 07:52:46.224001 8148 raft_consensus.cc:359] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:46.224737 8148 raft_consensus.cc:740] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872659a463ac417d9526c07b6f443c21, State: Initialized, Role: FOLLOWER
I20260807 07:52:46.225526 8148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:46.226433 8156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:46.226739 8156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:46.227595 8148 sys_catalog.cc:565] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:46.238158 8170 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:46.238262 8170 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: loading cluster ID for follower catalog manager: success
I20260807 07:52:46.242821 8170 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: acquiring CA information for follower catalog manager: success
I20260807 07:52:46.245654 8170 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.2.169.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:46.300642 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:44995
--webserver_interface=127.2.169.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:46.666962 8177 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:46.667296 8177 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:46.667383 8177 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:46.675737 8177 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:46.675843 8177 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:46.675900 8177 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:46.675995 8177 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:46.686992 8177 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:46.688622 8177 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:46.690111 8177 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:46.698370 8185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:46.699090 8182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:46.699730 8187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:46.700673 8177 server_base.cc:1061] running on GCE node
I20260807 07:52:46.701508 8177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:46.702947 8177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:46.704185 8177 hybrid_clock.cc:648] HybridClock initialized: now 1786089166704130 us; error 45 us; skew 500 ppm
I20260807 07:52:46.704540 8177 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:46 dist-test-slave-bw88 krb5kdc[8044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089166, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:46.708351 8177 init.cc:377] Logged in from keytab as kudu/127.2.169.253@KRBTEST.COM (short username kudu)
I20260807 07:52:46.710062 8177 webserver.cc:533] Webserver started at http://127.2.169.253:45579/ using document root <none> and password file <none>
I20260807 07:52:46.710801 8177 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:46.710914 8177 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:46.717195 8177 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 07:52:46.720172 8195 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:46.721640 8177 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:52:46.721782 8177 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "407070fcf89447f39beccb665abac23a"
format_stamp: "Formatted at 2026-08-07 07:52:37 on dist-test-slave-bw88"
I20260807 07:52:46.722504 8177 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:46.743086 8177 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:46.747241 8177 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:46.747546 8177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:46.769848 8177 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:44995
I20260807 07:52:46.769856 8263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:44995 every 8 connection(s)
I20260807 07:52:46.771579 8177 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:46.778069 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8177
I20260807 07:52:46.778299 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:52:46.779385 8264 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:46.784713 8264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Bootstrap starting.
WARNING: no policy specified for kudu/127.2.169.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260807 07:52:46.804656 8264 log.cc:826] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:46.818667 8264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:46.819259 8264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Bootstrap complete.
I20260807 07:52:46.826685 8264 raft_consensus.cc:359] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:46.827503 8264 raft_consensus.cc:740] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407070fcf89447f39beccb665abac23a, State: Initialized, Role: FOLLOWER
I20260807 07:52:46.828433 8264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:46.829497 8271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:46.829788 8271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:46.830575 8264 sys_catalog.cc:565] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.252 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.252 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:46.841727 8285 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:46.841887 8285 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: loading cluster ID for follower catalog manager: success
I20260807 07:52:46.846750 8285 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: acquiring CA information for follower catalog manager: success
I20260807 07:52:46.849695 8285 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.2.169.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.252 with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.252 with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:46.887465 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:33687
--webserver_interface=127.2.169.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:47.257243 8291 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:47.257695 8291 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:47.257862 8291 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:47.265985 8291 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:47.266146 8291 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:47.266281 8291 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:47.266358 8291 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:47.277228 8291 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--master_auto_join_cluster=false
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:33687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:47.279012 8291 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:47.280474 8291 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:47.288379 8299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:47.288468 8298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:47.289299 8301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:47.289970 8291 server_base.cc:1061] running on GCE node
I20260807 07:52:47.290709 8291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:47.292059 8291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:47.293318 8291 hybrid_clock.cc:648] HybridClock initialized: now 1786089167293269 us; error 50 us; skew 500 ppm
I20260807 07:52:47.293718 8291 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:47 dist-test-slave-bw88 krb5kdc[8044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089167, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:47.297298 8291 init.cc:377] Logged in from keytab as kudu/127.2.169.252@KRBTEST.COM (short username kudu)
I20260807 07:52:47.298913 8291 webserver.cc:533] Webserver started at http://127.2.169.252:42065/ using document root <none> and password file <none>
I20260807 07:52:47.299636 8291 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:47.299736 8291 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:47.305261 8291 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 07:52:47.308085 8310 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:47.309458 8291 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 07:52:47.309659 8291 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "c275a4ed29d74f89a2fd7fb95861867c"
format_stamp: "Formatted at 2026-08-07 07:52:39 on dist-test-slave-bw88"
I20260807 07:52:47.310364 8291 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:47.329761 8291 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:47.333735 8291 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:47.334007 8291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:47.354854 8291 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:33687
I20260807 07:52:47.354888 8373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:33687 every 8 connection(s)
I20260807 07:52:47.356469 8291 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:52:47.363593 8374 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:52:47.363648 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8291
I20260807 07:52:47.363896 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/wal/instance
I20260807 07:52:47.368844 8374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: Bootstrap starting.
Aug 07 07:52:47 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
I20260807 07:52:47.392541 8374 log.cc:826] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:47.406810 8374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
Aug 07 07:52:47 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:47.407516 8374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: Bootstrap complete.
I20260807 07:52:47.417593 8374 raft_consensus.cc:359] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:47.418689 8374 raft_consensus.cc:740] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c275a4ed29d74f89a2fd7fb95861867c, State: Initialized, Role: FOLLOWER
I20260807 07:52:47.419535 8374 consensus_queue.cc:260] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:47.420608 8382 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:47.420980 8382 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:47.421650 8374 sys_catalog.cc:565] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 07:52:47 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.252@KRBTEST.COM
I20260807 07:52:47.433851 8397 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:47.434022 8397 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: loading cluster ID for follower catalog manager: success
I20260807 07:52:47.438192 8397 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: acquiring CA information for follower catalog manager: success
I20260807 07:52:47.441051 8397 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:47.999405 8399 raft_consensus.cc:493] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:47.999707 8399 raft_consensus.cc:515] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:48.002135 8399 leader_election.cc:290] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 407070fcf89447f39beccb665abac23a (127.2.169.253:44995), c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687)
I20260807 07:52:48.023213 8343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c" is_pre_election: true
I20260807 07:52:48.023742 8343 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872659a463ac417d9526c07b6f443c21 in term 1.
I20260807 07:52:48.023968 8228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "407070fcf89447f39beccb665abac23a" is_pre_election: true
I20260807 07:52:48.024499 8228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872659a463ac417d9526c07b6f443c21 in term 1.
I20260807 07:52:48.024660 8084 leader_election.cc:304] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872659a463ac417d9526c07b6f443c21, c275a4ed29d74f89a2fd7fb95861867c; no voters:
I20260807 07:52:48.025079 8399 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 07:52:48.025229 8399 raft_consensus.cc:493] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:48.025411 8399 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:48.029420 8399 raft_consensus.cc:515] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:48.030112 8399 leader_election.cc:290] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 2 election: Requested vote from peers 407070fcf89447f39beccb665abac23a (127.2.169.253:44995), c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687)
I20260807 07:52:48.030782 8228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "407070fcf89447f39beccb665abac23a"
I20260807 07:52:48.030726 8343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c"
I20260807 07:52:48.030970 8343 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:48.030977 8228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:48.034898 8228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872659a463ac417d9526c07b6f443c21 in term 2.
I20260807 07:52:48.035069 8343 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872659a463ac417d9526c07b6f443c21 in term 2.
I20260807 07:52:48.035411 8086 leader_election.cc:304] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, 872659a463ac417d9526c07b6f443c21; no voters:
I20260807 07:52:48.035776 8399 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:48.036181 8399 raft_consensus.cc:697] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 LEADER]: Becoming Leader. State: Replica: 872659a463ac417d9526c07b6f443c21, State: Running, Role: LEADER
I20260807 07:52:48.036655 8399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:48.039770 8407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 872659a463ac417d9526c07b6f443c21. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.040092 8407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:48.040942 8410 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:48.043527 8410 catalog_manager.cc:679] Loaded metadata for table first_table [id=8aff27aa969f476aadecf95fcf0f4c03]
I20260807 07:52:48.045105 8410 tablet_loader.cc:96] loaded metadata for tablet b217bdb88a2b419692eb9b29e3c6c799 (table first_table [id=8aff27aa969f476aadecf95fcf0f4c03])
I20260807 07:52:48.045480 8410 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:48.046332 8410 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:48.046404 8410 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:48.047628 8410 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:48.048504 8410 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: Loaded TSK: 0
I20260807 07:52:48.049418 8410 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.169.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.2.169.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.169.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:52:48.171192 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:52:48.520267 8416 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:48.520632 8416 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:48.520805 8416 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:48.529726 8416 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:48.529939 8416 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:48.540374 8416 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.2.169.254:33341,127.2.169.253:44995,127.2.169.252:33687
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:48.542119 8416 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:48.543560 8416 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:48.553812 8425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:48.553833 8430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:48.554180 8422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:48.554752 8416 server_base.cc:1061] running on GCE node
I20260807 07:52:48.555380 8416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:48.556439 8416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:48.557696 8416 hybrid_clock.cc:648] HybridClock initialized: now 1786089168557652 us; error 54 us; skew 500 ppm
I20260807 07:52:48.558043 8416 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:52:48 dist-test-slave-bw88 krb5kdc[8044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:52:48.562014 8416 init.cc:377] Logged in from keytab as kudu/127.2.169.193@KRBTEST.COM (short username kudu)
I20260807 07:52:48.563711 8416 webserver.cc:533] Webserver started at http://127.2.169.193:33337/ using document root <none> and password file <none>
I20260807 07:52:48.564522 8416 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:48.564688 8416 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:48.570775 8416 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 07:52:48.573815 8439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:48.575234 8416 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 07:52:48.575461 8416 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "8983fadcb93943dc9c97956e233f8787"
format_stamp: "Formatted at 2026-08-07 07:52:38 on dist-test-slave-bw88"
I20260807 07:52:48.576218 8416 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:48.592541 8416 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:48.596608 8416 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:48.596843 8416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:48.598026 8416 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:48.600054 8228 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 2 FOLLOWER]: Refusing update from remote peer 872659a463ac417d9526c07b6f443c21: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260807 07:52:48.600464 8447 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20260807 07:52:48.601315 8445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 07:52:48.605583 8448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 872659a463ac417d9526c07b6f443c21. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.605891 8448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:48.607640 8445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.607901 8445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:48.608927 8343 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 2 FOLLOWER]: Refusing update from remote peer 872659a463ac417d9526c07b6f443c21: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260807 07:52:48.611402 8416 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 07:52:48.611541 8416 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.012s user 0.001s sys 0.000s
I20260807 07:52:48.611686 8416 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260807 07:52:48.616266 8448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.616508 8448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:48.617669 8416 ts_tablet_manager.cc:616] Registered 1 tablets
I20260807 07:52:48.617785 8416 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.006s sys 0.000s
I20260807 07:52:48.618772 8447 tablet_bootstrap.cc:492] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:48.627228 8456 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260807 07:52:48.637571 8460 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 872659a463ac417d9526c07b6f443c21. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.637996 8460 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:48.638036 8465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.638507 8465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:48.653172 8447 log.cc:826] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:48.660576 8447 tablet_bootstrap.cc:492] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:52:48.661736 8447 tablet_bootstrap.cc:492] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Bootstrap complete.
I20260807 07:52:48.664052 8447 ts_tablet_manager.cc:1403] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.046s user 0.015s sys 0.020s
I20260807 07:52:48.674888 8593 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:34761 every 8 connection(s)
I20260807 07:52:48.674908 8416 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:34761
I20260807 07:52:48.677033 8416 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:48.676520 8447 raft_consensus.cc:359] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:48.677670 8447 raft_consensus.cc:740] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:48.679040 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8416
I20260807 07:52:48.679291 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:52:48.679316 8447 consensus_queue.cc:260] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:48.679657 8447 raft_consensus.cc:399] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:48.679842 8447 raft_consensus.cc:493] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:48.680083 8447 raft_consensus.cc:3060] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:48.687883 8447 raft_consensus.cc:515] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:48.688884 8447 leader_election.cc:304] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:48.712438 8447 leader_election.cc:290] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:52:48.712523 8600 raft_consensus.cc:2804] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 2 FOLLOWER]: Leader election won for term 2
Aug 07 07:52:48 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for kudu/127.2.169.252@KRBTEST.COM
I20260807 07:52:48.724673 8600 raft_consensus.cc:697] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [term 2 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:48.725505 8600 consensus_queue.cc:237] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } }
I20260807 07:52:48.738130 8595 heartbeater.cc:344] Connected to a master server at 127.2.169.252:33687
I20260807 07:52:48.738575 8595 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:48.739753 8595 heartbeater.cc:507] Master 127.2.169.252:33687 requested a full tablet report, sending...
I20260807 07:52:48.750064 8333 ts_manager.cc:194] Registered new tserver with Master: 8983fadcb93943dc9c97956e233f8787 (127.2.169.193:34761)
I20260807 07:52:48.752815 8447 ts_tablet_manager.cc:1434] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.089s user 0.034s sys 0.049s
Aug 07 07:52:48 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:52:48 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.193@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:52:48.764909 8594 heartbeater.cc:344] Connected to a master server at 127.2.169.254:33341
I20260807 07:52:48.765094 8594 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:48.765617 8594 heartbeater.cc:507] Master 127.2.169.254:33341 requested a full tablet report, sending...
I20260807 07:52:48.768394 8104 ts_manager.cc:194] Registered new tserver with Master: 8983fadcb93943dc9c97956e233f8787 (127.2.169.193:34761)
I20260807 07:52:48.770155 8104 catalog_manager.cc:5697] T b217bdb88a2b419692eb9b29e3c6c799 P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 43449 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:48.778302 8596 heartbeater.cc:344] Connected to a master server at 127.2.169.253:44995
I20260807 07:52:48.778262 8445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "872659a463ac417d9526c07b6f443c21" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:48.778509 8445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:48.778585 8596 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:48.779307 8596 heartbeater.cc:507] Master 127.2.169.253:44995 requested a full tablet report, sending...
I20260807 07:52:48.782852 8215 ts_manager.cc:194] Registered new tserver with Master: 8983fadcb93943dc9c97956e233f8787 (127.2.169.193:34761)
I20260807 07:52:48.784407 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:48.796577 8104 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.169.193@KRBTEST.COM'} at 127.2.169.193:49775
I20260807 07:52:48.799399 8594 heartbeater.cc:499] Master 127.2.169.254:33341 was elected leader, sending a full tablet report...
I20260807 07:52:48.813431 2727 dynamic_multi_master-test.cc:649] Transferring leadership to master: c275a4ed29d74f89a2fd7fb95861867c
I20260807 07:52:48.817493 8118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "872659a463ac417d9526c07b6f443c21"
mode: GRACEFUL
new_leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47402
I20260807 07:52:48.817979 8118 raft_consensus.cc:606] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 LEADER]: Received request to transfer leadership to TS c275a4ed29d74f89a2fd7fb95861867c
I20260807 07:52:49.409904 8619 raft_consensus.cc:993] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21: : Instructing follower c275a4ed29d74f89a2fd7fb95861867c to start an election
I20260807 07:52:49.410140 8620 raft_consensus.cc:1081] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 LEADER]: Signalling peer c275a4ed29d74f89a2fd7fb95861867c to start an election
I20260807 07:52:49.411154 8343 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c"
from {username='kudu', principal='kudu/127.2.169.254@KRBTEST.COM'} at 127.0.0.1:35124
I20260807 07:52:49.411446 8343 raft_consensus.cc:493] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 07:52:49.411628 8343 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:49.413702 8343 raft_consensus.cc:515] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:49.415797 8343 leader_election.cc:290] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [CANDIDATE]: Term 3 election: Requested vote from peers 872659a463ac417d9526c07b6f443c21 (127.2.169.254:33341), 407070fcf89447f39beccb665abac23a (127.2.169.253:44995)
I20260807 07:52:49.429982 8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c275a4ed29d74f89a2fd7fb95861867c" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "872659a463ac417d9526c07b6f443c21"
I20260807 07:52:49.430305 8118 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 07:52:49.430445 8118 raft_consensus.cc:740] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 872659a463ac417d9526c07b6f443c21, State: Running, Role: LEADER
I20260807 07:52:49.430485 8228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c275a4ed29d74f89a2fd7fb95861867c" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "407070fcf89447f39beccb665abac23a"
I20260807 07:52:49.430804 8228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:49.430845 8118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:49.431440 8118 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:52:49.432986 8228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c275a4ed29d74f89a2fd7fb95861867c in term 3.
I20260807 07:52:49.433636 8118 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c275a4ed29d74f89a2fd7fb95861867c in term 3.
I20260807 07:52:49.433758 8314 leader_election.cc:304] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, c275a4ed29d74f89a2fd7fb95861867c; no voters:
I20260807 07:52:49.434587 8626 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:52:49.435030 8626 raft_consensus.cc:697] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 LEADER]: Becoming Leader. State: Replica: c275a4ed29d74f89a2fd7fb95861867c, State: Running, Role: LEADER
I20260807 07:52:49.435527 8626 consensus_queue.cc:237] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:49.436908 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c275a4ed29d74f89a2fd7fb95861867c. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.437117 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:49.437805 8628 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:49.440111 8628 catalog_manager.cc:679] Loaded metadata for table first_table [id=8aff27aa969f476aadecf95fcf0f4c03]
I20260807 07:52:49.441756 8628 tablet_loader.cc:96] loaded metadata for tablet b217bdb88a2b419692eb9b29e3c6c799 (table first_table [id=8aff27aa969f476aadecf95fcf0f4c03])
I20260807 07:52:49.442103 8628 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:49.442834 8628 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:49.442907 8628 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:49.443866 8628 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:49.444641 8628 catalog_manager.cc:6081] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c: Loaded TSK: 0
I20260807 07:52:49.445565 8628 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:49.878773 8228 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Refusing update from remote peer c275a4ed29d74f89a2fd7fb95861867c: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260807 07:52:49.879794 8627 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [LEADER]: Connected to new peer: Peer: permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260807 07:52:49.883453 8118 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Refusing update from remote peer c275a4ed29d74f89a2fd7fb95861867c: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260807 07:52:49.884433 8627 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [LEADER]: Connected to new peer: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260807 07:52:49.892531 8620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c275a4ed29d74f89a2fd7fb95861867c. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.892867 8620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:49.898916 8626 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.899269 8626 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:49.899549 8639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c275a4ed29d74f89a2fd7fb95861867c. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.899693 8620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.899842 8639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:49.899909 8620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:49.901121 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.901573 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:49.902520 8639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c275a4ed29d74f89a2fd7fb95861867c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:49.902726 8639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 07:52:50 dist-test-slave-bw88 krb5kdc[8044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.193@KRBTEST.COM
I20260807 07:52:50.079962 8511 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
407070fcf89447f39beccb665abac23a | 127.2.169.253:44995 | HEALTHY
872659a463ac417d9526c07b6f443c21 | 127.2.169.254:33341 | HEALTHY
c275a4ed29d74f89a2fd7fb95861867c | 127.2.169.252:33687 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb | hidden | 127.2.169.254:33341
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb | hidden | 127.2.169.253:44995
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb | hidden | 127.2.169.252:33687
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:45911 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
8983fadcb93943dc9c97956e233f8787 | 127.2.169.193:34761 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.2.169.193 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_3.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:45911 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 07:52:50.124704 2727 log_verifier.cc:126] Checking tablet b217bdb88a2b419692eb9b29e3c6c799
I20260807 07:52:50.136868 2727 log_verifier.cc:177] Verified matching terms for 2 ops in tablet b217bdb88a2b419692eb9b29e3c6c799
I20260807 07:52:50.137135 2727 dynamic_multi_master-test.cc:657] Verifying the first table
I20260807 07:52:50.185034 2727 dynamic_multi_master-test.cc:660] Creating and verifying the second table
I20260807 07:52:50.215354 8333 catalog_manager.cc:2283] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35170:
name: "second_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:50.257269 8511 tablet_service.cc:1511] Processing CreateTablet for tablet e401f49604ae4f1aa96ab220ddf806ac (DEFAULT_TABLE table=second_table [id=c695528f29214db6b9a8c3a85b6e2648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:50.258525 8511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e401f49604ae4f1aa96ab220ddf806ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:50.267715 8703 tablet_bootstrap.cc:492] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:50.269758 8703 tablet_bootstrap.cc:654] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:50.272271 8703 tablet_bootstrap.cc:492] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787: No bootstrap required, opened a new log
I20260807 07:52:50.272437 8703 ts_tablet_manager.cc:1403] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:52:50.273655 8703 raft_consensus.cc:359] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.273862 8703 raft_consensus.cc:385] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:50.274000 8703 raft_consensus.cc:740] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:50.274283 8703 consensus_queue.cc:260] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.274487 8703 raft_consensus.cc:399] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:50.274590 8703 raft_consensus.cc:493] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:50.274713 8703 raft_consensus.cc:3060] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:50.278031 8703 raft_consensus.cc:515] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.278354 8703 leader_election.cc:304] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:50.279006 8703 leader_election.cc:290] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:50.279253 8705 raft_consensus.cc:2804] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:50.279939 8705 raft_consensus.cc:697] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [term 1 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:50.280402 8705 consensus_queue.cc:237] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.280833 8595 heartbeater.cc:499] Master 127.2.169.252:33687 was elected leader, sending a full tablet report...
I20260807 07:52:50.286119 8703 ts_tablet_manager.cc:1434] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260807 07:52:50.287653 8333 catalog_manager.cc:5697] T e401f49604ae4f1aa96ab220ddf806ac P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8983fadcb93943dc9c97956e233f8787 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:50.364269 2727 dynamic_multi_master-test.cc:669] Pausing and resuming individual masters
I20260807 07:52:50.364372 2727 dynamic_multi_master-test.cc:673] Pausing master 872659a463ac417d9526c07b6f443c21, 127.2.169.254:33341
I20260807 07:52:50.431819 8331 catalog_manager.cc:2283] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35194:
name: "table-0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:50.443950 8511 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4bef793c684412aa59347fa1ff71ca (DEFAULT_TABLE table=table-0 [id=a638d9ceb6084e1499061f5b42acafe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:50.444559 8511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4bef793c684412aa59347fa1ff71ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:50.451674 8703 tablet_bootstrap.cc:492] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:50.453706 8703 tablet_bootstrap.cc:654] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:50.455785 8703 tablet_bootstrap.cc:492] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787: No bootstrap required, opened a new log
I20260807 07:52:50.455972 8703 ts_tablet_manager.cc:1403] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:52:50.456693 8703 raft_consensus.cc:359] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.456909 8703 raft_consensus.cc:385] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:50.457013 8703 raft_consensus.cc:740] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:50.457497 8703 consensus_queue.cc:260] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.457716 8703 raft_consensus.cc:399] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:50.457859 8703 raft_consensus.cc:493] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:50.457989 8703 raft_consensus.cc:3060] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:50.460075 8703 raft_consensus.cc:515] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.460281 8703 leader_election.cc:304] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:50.460510 8703 leader_election.cc:290] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:50.460584 8708 raft_consensus.cc:2804] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:50.460760 8708 raft_consensus.cc:697] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [term 1 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:50.460970 8703 ts_tablet_manager.cc:1434] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 07:52:50.461077 8708 consensus_queue.cc:237] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.464323 8331 catalog_manager.cc:5697] T 7d4bef793c684412aa59347fa1ff71ca P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8983fadcb93943dc9c97956e233f8787 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:50.481622 2727 dynamic_multi_master-test.cc:673] Pausing master 407070fcf89447f39beccb665abac23a, 127.2.169.253:44995
I20260807 07:52:50.571784 8331 catalog_manager.cc:2283] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35218:
name: "table-1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:50.584215 8511 tablet_service.cc:1511] Processing CreateTablet for tablet d4c148529591498c8d1812b717e967cf (DEFAULT_TABLE table=table-1 [id=303f52f537d145569173500f30af37a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:50.584769 8511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4c148529591498c8d1812b717e967cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:50.590579 8703 tablet_bootstrap.cc:492] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:50.592638 8703 tablet_bootstrap.cc:654] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:50.595084 8703 tablet_bootstrap.cc:492] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787: No bootstrap required, opened a new log
I20260807 07:52:50.595259 8703 ts_tablet_manager.cc:1403] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:52:50.596235 8703 raft_consensus.cc:359] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.596442 8703 raft_consensus.cc:385] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:50.596606 8703 raft_consensus.cc:740] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:50.596987 8703 consensus_queue.cc:260] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.597307 8703 raft_consensus.cc:399] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:50.597507 8703 raft_consensus.cc:493] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:50.597662 8703 raft_consensus.cc:3060] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:50.600978 8703 raft_consensus.cc:515] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.601313 8703 leader_election.cc:304] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:50.601613 8703 leader_election.cc:290] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:50.601708 8708 raft_consensus.cc:2804] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:50.601938 8708 raft_consensus.cc:697] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [term 1 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:50.602708 8708 consensus_queue.cc:237] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:50.603045 8703 ts_tablet_manager.cc:1434] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260807 07:52:50.606016 8331 catalog_manager.cc:5697] T d4c148529591498c8d1812b717e967cf P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8983fadcb93943dc9c97956e233f8787 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:50.630345 2727 dynamic_multi_master-test.cc:673] Pausing master c275a4ed29d74f89a2fd7fb95861867c, 127.2.169.252:33687
I20260807 07:52:52.119722 8775 raft_consensus.cc:493] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c275a4ed29d74f89a2fd7fb95861867c)
I20260807 07:52:52.119977 8775 raft_consensus.cc:515] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:52.120796 8775 leader_election.cc:290] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 407070fcf89447f39beccb665abac23a (127.2.169.253:44995), c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687)
I20260807 07:52:52.121434 8228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "407070fcf89447f39beccb665abac23a" is_pre_election: true
I20260807 07:52:52.131865 8776 raft_consensus.cc:493] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c275a4ed29d74f89a2fd7fb95861867c)
I20260807 07:52:52.132045 8776 raft_consensus.cc:515] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:52.134111 8776 leader_election.cc:290] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 872659a463ac417d9526c07b6f443c21 (127.2.169.254:33341), c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687)
I20260807 07:52:52.141557 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "872659a463ac417d9526c07b6f443c21" is_pre_election: true
I20260807 07:52:52.141757 8117 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407070fcf89447f39beccb665abac23a in term 3.
I20260807 07:52:52.142372 8199 leader_election.cc:304] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, 872659a463ac417d9526c07b6f443c21; no voters:
I20260807 07:52:52.142724 8776 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 07:52:52.142835 8776 raft_consensus.cc:493] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Starting leader election (detected failure of leader c275a4ed29d74f89a2fd7fb95861867c)
I20260807 07:52:52.142918 8776 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:52.145094 8776 raft_consensus.cc:515] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:52.145751 8776 leader_election.cc:290] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 4 election: Requested vote from peers 872659a463ac417d9526c07b6f443c21 (127.2.169.254:33341), c275a4ed29d74f89a2fd7fb95861867c (127.2.169.252:33687)
I20260807 07:52:52.146144 8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "872659a463ac417d9526c07b6f443c21"
I20260807 07:52:52.146341 8117 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:52.148329 8117 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407070fcf89447f39beccb665abac23a in term 4.
I20260807 07:52:52.148730 8199 leader_election.cc:304] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407070fcf89447f39beccb665abac23a, 872659a463ac417d9526c07b6f443c21; no voters:
I20260807 07:52:52.149048 8776 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 4 FOLLOWER]: Leader election won for term 4
I20260807 07:52:52.149477 8776 raft_consensus.cc:697] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [term 4 LEADER]: Becoming Leader. State: Replica: 407070fcf89447f39beccb665abac23a, State: Running, Role: LEADER
I20260807 07:52:52.149916 8776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:52.151655 8780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407070fcf89447f39beccb665abac23a. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:52.151991 8780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:52.152673 8784 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:52.155148 8784 catalog_manager.cc:679] Loaded metadata for table table-1 [id=303f52f537d145569173500f30af37a7]
I20260807 07:52:52.155614 8784 catalog_manager.cc:679] Loaded metadata for table first_table [id=8aff27aa969f476aadecf95fcf0f4c03]
I20260807 07:52:52.155968 8784 catalog_manager.cc:679] Loaded metadata for table table-0 [id=a638d9ceb6084e1499061f5b42acafe4]
I20260807 07:52:52.156345 8784 catalog_manager.cc:679] Loaded metadata for table second_table [id=c695528f29214db6b9a8c3a85b6e2648]
I20260807 07:52:52.157976 8784 tablet_loader.cc:96] loaded metadata for tablet 7d4bef793c684412aa59347fa1ff71ca (table table-0 [id=a638d9ceb6084e1499061f5b42acafe4])
I20260807 07:52:52.158368 8784 tablet_loader.cc:96] loaded metadata for tablet b217bdb88a2b419692eb9b29e3c6c799 (table first_table [id=8aff27aa969f476aadecf95fcf0f4c03])
I20260807 07:52:52.158687 8784 tablet_loader.cc:96] loaded metadata for tablet d4c148529591498c8d1812b717e967cf (table table-1 [id=303f52f537d145569173500f30af37a7])
I20260807 07:52:52.159009 8784 tablet_loader.cc:96] loaded metadata for tablet e401f49604ae4f1aa96ab220ddf806ac (table second_table [id=c695528f29214db6b9a8c3a85b6e2648])
I20260807 07:52:52.159272 8784 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:52.160094 8784 catalog_manager.cc:1295] Loaded cluster ID: e27a52b2430a4a8db3a9b6a5502c434b
I20260807 07:52:52.160205 8784 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:52.161402 8784 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:52.162304 8784 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a: Loaded TSK: 0
I20260807 07:52:52.163117 8784 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:52.670296 8117 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 4 FOLLOWER]: Refusing update from remote peer 407070fcf89447f39beccb665abac23a: Log matching property violated. Preceding OpId in replica: term: 3 index: 23. Preceding OpId from leader: term: 4 index: 24. (index mismatch)
I20260807 07:52:52.671096 8780 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260807 07:52:52.673260 8785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407070fcf89447f39beccb665abac23a. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:52.673508 8785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:52.675529 8787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:52.675807 8787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:52.676590 8785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:52.676913 8785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:53.636689 8596 heartbeater.cc:499] Master 127.2.169.253:44995 was elected leader, sending a full tablet report...
W20260807 07:52:53.799413 8774 negotiation.cc:336] Failed RPC negotiation. Trace:
0807 07:52:50.655767 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.169.252:33687 (local address 127.0.0.1:35232)
0807 07:52:50.667923 (+ 12156us) negotiation.cc:107] Waiting for socket to connect
0807 07:52:50.667943 (+ 20us) client_negotiation.cc:175] Beginning negotiation
0807 07:52:50.668062 (+ 119us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0807 07:52:53.797717 (+3129655us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.2.169.252:33687: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":12026,"thread_start_us":12383,"threads_started":1}
I20260807 07:52:53.849313 8214 catalog_manager.cc:2283] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33016:
name: "table-2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:53.876596 8511 tablet_service.cc:1511] Processing CreateTablet for tablet 3ec812b8faed43e48de8013a1fd5aefc (DEFAULT_TABLE table=table-2 [id=34ec781db3d8410093e36996a7a30a74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:53.877127 8511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ec812b8faed43e48de8013a1fd5aefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:53.882340 8819 tablet_bootstrap.cc:492] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787: Bootstrap starting.
I20260807 07:52:53.883949 8819 tablet_bootstrap.cc:654] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:53.885946 8819 tablet_bootstrap.cc:492] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787: No bootstrap required, opened a new log
I20260807 07:52:53.886112 8819 ts_tablet_manager.cc:1403] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:52:53.886987 8819 raft_consensus.cc:359] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:53.887189 8819 raft_consensus.cc:385] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:53.887295 8819 raft_consensus.cc:740] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Initialized, Role: FOLLOWER
I20260807 07:52:53.887552 8819 consensus_queue.cc:260] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:53.887714 8819 raft_consensus.cc:399] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:53.887835 8819 raft_consensus.cc:493] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:53.887948 8819 raft_consensus.cc:3060] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:53.889926 8819 raft_consensus.cc:515] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:53.890180 8819 leader_election.cc:304] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8983fadcb93943dc9c97956e233f8787; no voters:
I20260807 07:52:53.890614 8819 leader_election.cc:290] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:53.890817 8821 raft_consensus.cc:2804] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:53.891364 8821 raft_consensus.cc:697] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [term 1 LEADER]: Becoming Leader. State: Replica: 8983fadcb93943dc9c97956e233f8787, State: Running, Role: LEADER
I20260807 07:52:53.891446 8819 ts_tablet_manager.cc:1434] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 07:52:53.891764 8821 consensus_queue.cc:237] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } }
I20260807 07:52:53.895442 8214 catalog_manager.cc:5697] T 3ec812b8faed43e48de8013a1fd5aefc P 8983fadcb93943dc9c97956e233f8787 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8983fadcb93943dc9c97956e233f8787 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "8983fadcb93943dc9c97956e233f8787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8983fadcb93943dc9c97956e233f8787" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 34761 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:53.902609 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8416
I20260807 07:52:53.905141 8343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "872659a463ac417d9526c07b6f443c21" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c" is_pre_election: true
W20260807 07:52:53.907800 8635 negotiation.cc:336] Failed RPC negotiation. Trace:
0807 07:52:53.903983 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.2.169.252:33687)
0807 07:52:53.906187 (+ 2204us) server_negotiation.cc:207] Beginning negotiation
0807 07:52:53.906194 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0807 07:52:53.906216 (+ 22us) server_negotiation.cc:408] Connection header received
0807 07:52:53.906290 (+ 74us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0807 07:52:53.906295 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0807 07:52:53.906384 (+ 89us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0807 07:52:53.906599 (+ 215us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35232: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2084}
I20260807 07:52:53.910024 8084 leader_election.cc:304] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 872659a463ac417d9526c07b6f443c21; no voters: 407070fcf89447f39beccb665abac23a, c275a4ed29d74f89a2fd7fb95861867c
I20260807 07:52:53.929869 8589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:53.942436 8831 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 07:52:53.944429 8117 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 872659a463ac417d9526c07b6f443c21 [term 4 FOLLOWER]: Rejecting Update request from peer c275a4ed29d74f89a2fd7fb95861867c for earlier term 3. Current term is 4. Ops: []
I20260807 07:52:53.945199 8627 consensus_queue.cc:1059] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 }, Status: INVALID_TERM, Last received: 3.23, Next index: 24, Last known committed idx: 27, Time since last communication: 0.000s
I20260807 07:52:53.945796 8626 raft_consensus.cc:3055] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 LEADER]: Stepping down as leader of term 3
I20260807 07:52:53.945940 8626 raft_consensus.cc:740] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c275a4ed29d74f89a2fd7fb95861867c, State: Running, Role: LEADER
I20260807 07:52:53.946300 8626 consensus_queue.cc:260] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } }
I20260807 07:52:53.947278 8626 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 3 FOLLOWER]: Advancing to term 4
I20260807 07:52:53.951766 8343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c" is_pre_election: true
I20260807 07:52:53.952119 8343 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407070fcf89447f39beccb665abac23a in term 4.
I20260807 07:52:53.952737 8343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "407070fcf89447f39beccb665abac23a" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "c275a4ed29d74f89a2fd7fb95861867c"
I20260807 07:52:53.953073 8342 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [term 4 FOLLOWER]: Refusing update from remote peer 407070fcf89447f39beccb665abac23a: Log matching property violated. Preceding OpId in replica: term: 3 index: 23. Preceding OpId from leader: term: 4 index: 24. (index mismatch)
I20260807 07:52:53.954874 8814 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 407070fcf89447f39beccb665abac23a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260807 07:52:53.970276 8626 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407070fcf89447f39beccb665abac23a. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:53.970546 8626 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:53.970789 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "407070fcf89447f39beccb665abac23a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "872659a463ac417d9526c07b6f443c21" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 33341 } } peers { permanent_uuid: "407070fcf89447f39beccb665abac23a" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 44995 } } peers { permanent_uuid: "c275a4ed29d74f89a2fd7fb95861867c" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 33687 } attrs { promote: false } } }
I20260807 07:52:53.970989 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P c275a4ed29d74f89a2fd7fb95861867c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:53.973323 8833 mvcc.cc:204] Tried to move back new op lower bound from 7315821256282193920 to 7315821249129664512. Current Snapshot: MvccSnapshot[applied={T|T < 7315821256282193920}]
I20260807 07:52:54.096109 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8416
I20260807 07:52:54.128845 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8063
I20260807 07:52:54.130067 8139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:54.254175 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8063
I20260807 07:52:54.287173 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8177
I20260807 07:52:54.288473 8256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:54.412019 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8177
I20260807 07:52:54.445289 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8291
I20260807 07:52:54.446485 8367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:54.567770 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8291
2026-08-07T07:52:54Z chronyd exiting
[ OK ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/3 (20366 ms)
[----------] 1 test from ParameterizedAddMasterTest (20366 ms total)
[----------] 1 test from ParameterizedRemoveMasterTest
[ RUN ] ParameterizedRemoveMasterTest.TestRemoveMaster/1
2026-08-07T07:52:54Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:52:54Z Disabled control of system clock
I20260807 07:52:54.668807 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:37689
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:37689,127.2.169.253:32963
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260807 07:52:55.022780 8845 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:55.023141 8845 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:55.023259 8845 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:55.031795 8845 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:55.031890 8845 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:55.031977 8845 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:55.032079 8845 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:55.042387 8845 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:37689,127.2.169.253:32963
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:37689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:55.042752 8845 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:37689,127.2.169.253:32963'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:55.044164 8845 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:55.046082 8845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:55.054649 8851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:55.054622 8852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:55.055290 8854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:55.055784 8845 server_base.cc:1061] running on GCE node
I20260807 07:52:55.056617 8845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:55.058118 8845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:55.059353 8845 hybrid_clock.cc:648] HybridClock initialized: now 1786089175059283 us; error 33 us; skew 500 ppm
I20260807 07:52:55.059723 8845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:55.062026 8845 webserver.cc:533] Webserver started at http://127.2.169.254:42047/ using document root <none> and password file <none>
I20260807 07:52:55.062764 8845 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:55.062908 8845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:55.063285 8845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:55.065827 8845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.066638 8845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.072192 8845 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 07:52:55.075700 8860 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:55.077160 8845 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 07:52:55.077469 8845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.077708 8845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:55.116866 8845 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:55.117858 8845 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:55.118206 8845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:55.140229 8845 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:37689
I20260807 07:52:55.140231 8924 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:37689 every 8 connection(s)
I20260807 07:52:55.142045 8845 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:52:55.144295 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8845
I20260807 07:52:55.144569 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:52:55.146354 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:32963
--webserver_interface=127.2.169.253
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:37689,127.2.169.253:32963
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260807 07:52:55.147645 8925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:55.156527 8925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:55.174561 8925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:52:55.175953 8864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
W20260807 07:52:55.178042 8925 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:32963: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
I20260807 07:52:55.227422 8925 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } attempt: 1
W20260807 07:52:55.229713 8925 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:32963: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
I20260807 07:52:55.297987 8925 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } attempt: 2
W20260807 07:52:55.300343 8925 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:32963: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
I20260807 07:52:55.391574 8925 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } attempt: 3
W20260807 07:52:55.393719 8925 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:32963: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
W20260807 07:52:55.519718 8927 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:55.520118 8927 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:55.520283 8927 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:55.528537 8927 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:52:55.528681 8927 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:55.528822 8927 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:52:55.528935 8927 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:52:55.537012 8925 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } attempt: 4
W20260807 07:52:55.539258 8925 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:32963: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
I20260807 07:52:55.539819 8927 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:37689,127.2.169.253:32963
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:32963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:52:55.540225 8927 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:37689,127.2.169.253:32963'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:52:55.541839 8927 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:55.543743 8927 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:55.552656 8939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:55.553043 8942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:55.552712 8940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:55.553891 8927 server_base.cc:1061] running on GCE node
I20260807 07:52:55.554786 8927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:55.556255 8927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:55.557467 8927 hybrid_clock.cc:648] HybridClock initialized: now 1786089175557386 us; error 38 us; skew 500 ppm
I20260807 07:52:55.557845 8927 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:55.560106 8927 webserver.cc:533] Webserver started at http://127.2.169.253:35493/ using document root <none> and password file <none>
I20260807 07:52:55.560864 8927 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:55.561022 8927 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:55.561507 8927 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:55.563993 8927 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "13f7fb477c3849f59b5dc83f46b6e841"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.564875 8927 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "13f7fb477c3849f59b5dc83f46b6e841"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.570622 8927 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 07:52:55.574330 8949 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:55.575968 8927 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 07:52:55.576169 8927 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "13f7fb477c3849f59b5dc83f46b6e841"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:52:55.576390 8927 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:55.594257 8927 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:55.595136 8927 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:55.595497 8927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:55.618705 8927 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:32963
I20260807 07:52:55.618726 9009 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:32963 every 8 connection(s)
I20260807 07:52:55.620398 8927 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:52:55.624807 9010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:55.628685 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 8927
I20260807 07:52:55.628897 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:52:55.631376 9010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:55.645954 9010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:52:55.656284 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: Bootstrap starting.
I20260807 07:52:55.659920 9010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:55.661500 9010 log.cc:826] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:55.665325 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: No bootstrap required, opened a new log
I20260807 07:52:55.670715 9010 raft_consensus.cc:359] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.671039 9010 raft_consensus.cc:385] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:55.671171 9010 raft_consensus.cc:740] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f7fb477c3849f59b5dc83f46b6e841, State: Initialized, Role: FOLLOWER
I20260807 07:52:55.671900 9010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.672791 9018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.673028 9018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:55.673825 9010 sys_catalog.cc:565] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:52:55.683884 9034 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:55.684005 9034 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:55.800674 9018 raft_consensus.cc:493] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:52:55.801005 9018 raft_consensus.cc:515] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.802537 9018 leader_election.cc:290] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5117ca209af493b8f88ebb7d0b7f9d2 (127.2.169.254:37689)
I20260807 07:52:55.838538 8925 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } attempt: 5
W20260807 07:52:55.847867 8955 tablet.cc:2406] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:52:55.851460 8925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Bootstrap starting.
I20260807 07:52:55.855682 8925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:55.857051 8925 log.cc:826] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:55.860455 8925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: No bootstrap required, opened a new log
I20260807 07:52:55.868219 8925 raft_consensus.cc:359] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.868571 8925 raft_consensus.cc:385] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:55.868682 8925 raft_consensus.cc:740] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117ca209af493b8f88ebb7d0b7f9d2, State: Initialized, Role: FOLLOWER
I20260807 07:52:55.869463 8925 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.870488 9037 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.870873 9037 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:55.871719 8925 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:52:55.871786 8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13f7fb477c3849f59b5dc83f46b6e841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" is_pre_election: true
I20260807 07:52:55.872329 8895 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f7fb477c3849f59b5dc83f46b6e841 in term 0.
I20260807 07:52:55.872992 8950 leader_election.cc:304] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 13f7fb477c3849f59b5dc83f46b6e841, b5117ca209af493b8f88ebb7d0b7f9d2; no voters:
I20260807 07:52:55.873457 9018 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 07:52:55.873624 9018 raft_consensus.cc:493] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:52:55.873775 9018 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:55.876456 9018 raft_consensus.cc:515] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.876977 9018 leader_election.cc:290] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [CANDIDATE]: Term 1 election: Requested vote from peers b5117ca209af493b8f88ebb7d0b7f9d2 (127.2.169.254:37689)
I20260807 07:52:55.877599 8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13f7fb477c3849f59b5dc83f46b6e841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
I20260807 07:52:55.877851 8895 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:55.880272 8895 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f7fb477c3849f59b5dc83f46b6e841 in term 1.
I20260807 07:52:55.880744 8950 leader_election.cc:304] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 13f7fb477c3849f59b5dc83f46b6e841, b5117ca209af493b8f88ebb7d0b7f9d2; no voters:
I20260807 07:52:55.881217 9018 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:55.881796 9018 raft_consensus.cc:697] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 LEADER]: Becoming Leader. State: Replica: 13f7fb477c3849f59b5dc83f46b6e841, State: Running, Role: LEADER
I20260807 07:52:55.882364 9018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:55.886722 9044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13f7fb477c3849f59b5dc83f46b6e841. Latest consensus state: current_term: 1 leader_uuid: "13f7fb477c3849f59b5dc83f46b6e841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.887014 9044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:55.891976 9048 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:55.894145 9048 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:55.902724 8895 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 1 FOLLOWER]: Refusing update from remote peer 13f7fb477c3849f59b5dc83f46b6e841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:52:55.903863 9044 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 07:52:55.904726 9055 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:52:55.904922 9055 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:52:55.913971 9037 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13f7fb477c3849f59b5dc83f46b6e841. Latest consensus state: current_term: 1 leader_uuid: "13f7fb477c3849f59b5dc83f46b6e841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.914208 9037 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:55.916671 9056 mvcc.cc:204] Tried to move back new op lower bound from 7315821264486010880 to 7315821264418734080. Current Snapshot: MvccSnapshot[applied={T|T < 7315821264486010880}]
I20260807 07:52:55.921650 9037 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13f7fb477c3849f59b5dc83f46b6e841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.921854 9037 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:55.923193 9044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13f7fb477c3849f59b5dc83f46b6e841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:55.923518 9044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:55.926936 9048 catalog_manager.cc:1383] Generated new cluster ID: be2a12a592854eb5a47023448481ede9
I20260807 07:52:55.927057 9048 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:55.998642 9048 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:52:56.000377 9048 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:56.017437 9048 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: Generated new TSK 0
I20260807 07:52:56.018533 9048 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:56.049048 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:37689,127.2.169.253:32963
--builtin_ntp_servers=127.2.169.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260807 07:52:56.434818 9063 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:52:56.435091 9063 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:52:56.435163 9063 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:52:56.443393 9063 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:52:56.443537 9063 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:52:56.453220 9063 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:37689,127.2.169.253:32963
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:52:56.454723 9063 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:52:56.456534 9063 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:52:56.466974 9071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:56.467497 9070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:52:56.467649 9073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:52:56.468243 9063 server_base.cc:1061] running on GCE node
I20260807 07:52:56.468840 9063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:52:56.469794 9063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:52:56.470997 9063 hybrid_clock.cc:648] HybridClock initialized: now 1786089176470953 us; error 37 us; skew 500 ppm
I20260807 07:52:56.471312 9063 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:52:56.473716 9063 webserver.cc:533] Webserver started at http://127.2.169.193:39281/ using document root <none> and password file <none>
I20260807 07:52:56.474488 9063 fs_manager.cc:362] Metadata directory not provided
I20260807 07:52:56.474586 9063 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:52:56.475082 9063 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:52:56.477929 9063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "186e245ca80c433fb555c492e2050e90"
format_stamp: "Formatted at 2026-08-07 07:52:56 on dist-test-slave-bw88"
I20260807 07:52:56.478752 9063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "186e245ca80c433fb555c492e2050e90"
format_stamp: "Formatted at 2026-08-07 07:52:56 on dist-test-slave-bw88"
I20260807 07:52:56.484412 9063 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 07:52:56.488133 9082 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:56.489658 9063 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 07:52:56.489866 9063 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "186e245ca80c433fb555c492e2050e90"
format_stamp: "Formatted at 2026-08-07 07:52:56 on dist-test-slave-bw88"
I20260807 07:52:56.490085 9063 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:52:56.512486 9063 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:52:56.513490 9063 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:52:56.513849 9063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:52:56.514967 9063 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:52:56.516956 9063 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:52:56.517082 9063 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:56.517222 9063 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:52:56.517329 9063 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:52:56.555984 9063 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:37499
I20260807 07:52:56.556080 9223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:37499 every 8 connection(s)
I20260807 07:52:56.557677 9063 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:52:56.563858 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9063
I20260807 07:52:56.564172 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:52:56.585634 9225 heartbeater.cc:344] Connected to a master server at 127.2.169.254:37689
I20260807 07:52:56.586042 9225 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:56.587049 9225 heartbeater.cc:507] Master 127.2.169.254:37689 requested a full tablet report, sending...
I20260807 07:52:56.587863 9224 heartbeater.cc:344] Connected to a master server at 127.2.169.253:32963
I20260807 07:52:56.588009 9224 heartbeater.cc:461] Registering TS with master...
I20260807 07:52:56.588505 9224 heartbeater.cc:507] Master 127.2.169.253:32963 requested a full tablet report, sending...
I20260807 07:52:56.589680 8883 ts_manager.cc:194] Registered new tserver with Master: 186e245ca80c433fb555c492e2050e90 (127.2.169.193:37499)
I20260807 07:52:56.590257 8968 ts_manager.cc:194] Registered new tserver with Master: 186e245ca80c433fb555c492e2050e90 (127.2.169.193:37499)
I20260807 07:52:56.591917 8968 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:45251
I20260807 07:52:56.601795 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:56.607594 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:52:56.617102 2727 dynamic_multi_master-test.cc:943] Transferring leadership to master: b5117ca209af493b8f88ebb7d0b7f9d2
I20260807 07:52:56.620349 8981 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "13f7fb477c3849f59b5dc83f46b6e841"
mode: GRACEFUL
new_leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
from {username='slave'} at 127.0.0.1:56222
I20260807 07:52:56.620702 8981 raft_consensus.cc:606] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 LEADER]: Received request to transfer leadership to TS b5117ca209af493b8f88ebb7d0b7f9d2
I20260807 07:52:56.906881 9055 catalog_manager.cc:1295] Loaded cluster ID: be2a12a592854eb5a47023448481ede9
I20260807 07:52:56.906991 9055 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: loading cluster ID for follower catalog manager: success
I20260807 07:52:56.909711 9055 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: acquiring CA information for follower catalog manager: success
I20260807 07:52:56.911535 9055 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:52:56.967331 9097 raft_consensus.cc:993] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841: : Instructing follower b5117ca209af493b8f88ebb7d0b7f9d2 to start an election
I20260807 07:52:56.967537 9097 raft_consensus.cc:1081] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 LEADER]: Signalling peer b5117ca209af493b8f88ebb7d0b7f9d2 to start an election
I20260807 07:52:56.968533 8895 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
from {username='slave'} at 127.0.0.1:49202
I20260807 07:52:56.968770 8895 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 07:52:56.968878 8895 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:56.970927 8895 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:56.971949 8895 leader_election.cc:290] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [CANDIDATE]: Term 2 election: Requested vote from peers 13f7fb477c3849f59b5dc83f46b6e841 (127.2.169.253:32963)
I20260807 07:52:56.972694 8981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "13f7fb477c3849f59b5dc83f46b6e841"
I20260807 07:52:56.972959 8981 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 07:52:56.973064 8981 raft_consensus.cc:740] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13f7fb477c3849f59b5dc83f46b6e841, State: Running, Role: LEADER
I20260807 07:52:56.973459 8981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:56.973860 8981 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:52:56.975888 8981 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5117ca209af493b8f88ebb7d0b7f9d2 in term 2.
I20260807 07:52:56.976389 8864 leader_election.cc:304] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 13f7fb477c3849f59b5dc83f46b6e841, b5117ca209af493b8f88ebb7d0b7f9d2; no voters:
I20260807 07:52:56.977149 9232 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:52:56.977672 9232 raft_consensus.cc:697] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 LEADER]: Becoming Leader. State: Replica: b5117ca209af493b8f88ebb7d0b7f9d2, State: Running, Role: LEADER
I20260807 07:52:56.978075 9232 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } }
I20260807 07:52:56.979276 9233 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5117ca209af493b8f88ebb7d0b7f9d2. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:56.979521 9233 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:56.980096 9235 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:52:56.981828 9235 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:52:56.982661 9235 catalog_manager.cc:1295] Loaded cluster ID: be2a12a592854eb5a47023448481ede9
I20260807 07:52:56.982759 9235 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:52:56.984010 9235 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:52:56.984697 9235 catalog_manager.cc:6081] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Loaded TSK: 0
I20260807 07:52:56.985567 9235 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:52:57.188169 8883 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49204:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:52:57.198004 8981 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [term 2 FOLLOWER]: Refusing update from remote peer b5117ca209af493b8f88ebb7d0b7f9d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260807 07:52:57.198807 9233 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 07:52:57.202078 9097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5117ca209af493b8f88ebb7d0b7f9d2. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:57.202315 9097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:57.203896 9233 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:57.204149 9233 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
I20260807 07:52:57.204921 9097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:57.205134 9097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f7fb477c3849f59b5dc83f46b6e841 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:52:57.223949 9145 tablet_service.cc:1511] Processing CreateTablet for tablet 355f713713c14c9bbc4febb329e0a746 (DEFAULT_TABLE table=first_table [id=6117cb77e8684811be5c57c2b5957025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:52:57.225312 9145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355f713713c14c9bbc4febb329e0a746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:52:57.235159 9255 tablet_bootstrap.cc:492] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Bootstrap starting.
I20260807 07:52:57.238269 9255 tablet_bootstrap.cc:654] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Neither blocks nor log segments found. Creating new log.
I20260807 07:52:57.239334 9255 log.cc:826] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Log is configured to *not* fsync() on all Append() calls
I20260807 07:52:57.242141 9255 tablet_bootstrap.cc:492] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: No bootstrap required, opened a new log
I20260807 07:52:57.242450 9255 ts_tablet_manager.cc:1403] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 07:52:57.247432 9255 raft_consensus.cc:359] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:52:57.247720 9255 raft_consensus.cc:385] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:52:57.247797 9255 raft_consensus.cc:740] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186e245ca80c433fb555c492e2050e90, State: Initialized, Role: FOLLOWER
I20260807 07:52:57.248379 9255 consensus_queue.cc:260] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:52:57.248569 9255 raft_consensus.cc:399] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:52:57.248667 9255 raft_consensus.cc:493] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:52:57.248785 9255 raft_consensus.cc:3060] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:52:57.251067 9255 raft_consensus.cc:515] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:52:57.251530 9255 leader_election.cc:304] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186e245ca80c433fb555c492e2050e90; no voters:
I20260807 07:52:57.252116 9255 leader_election.cc:290] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:52:57.252300 9258 raft_consensus.cc:2804] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:52:57.253010 9258 raft_consensus.cc:697] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 LEADER]: Becoming Leader. State: Replica: 186e245ca80c433fb555c492e2050e90, State: Running, Role: LEADER
I20260807 07:52:57.253284 9224 heartbeater.cc:499] Master 127.2.169.253:32963 was elected leader, sending a full tablet report...
I20260807 07:52:57.253618 9258 consensus_queue.cc:237] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:52:57.254024 9255 ts_tablet_manager.cc:1434] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 07:52:57.260905 8883 catalog_manager.cc:5697] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 reported cstate change: term changed from 0 to 1, leader changed from <none> to 186e245ca80c433fb555c492e2050e90 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "186e245ca80c433fb555c492e2050e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } health_report { overall_health: HEALTHY } } }
I20260807 07:52:57.267915 9225 heartbeater.cc:499] Master 127.2.169.254:37689 was elected leader, sending a full tablet report...
I20260807 07:52:57.277144 2727 dynamic_multi_master-test.cc:961] Shutting down the master to be removed
I20260807 07:52:57.277261 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8927
I20260807 07:52:57.278343 9005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:52:57.385497 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8927
I20260807 07:52:57.414486 2727 dynamic_multi_master-test.cc:963] Detecting transition to terminal FAILED state
W20260807 07:52:57.729321 8864 consensus_peers.cc:597] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 -> Peer 13f7fb477c3849f59b5dc83f46b6e841 (127.2.169.253:32963): Couldn't send request to peer 13f7fb477c3849f59b5dc83f46b6e841. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 07:52:58.124884 9233 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:52:58.125240 9233 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
W20260807 07:52:58.264874 9224 heartbeater.cc:646] Failed to heartbeat to 127.2.169.253:32963 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111)
W20260807 07:53:00.296083 8864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260807 07:53:00.298331 8864 consensus_peers.cc:597] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 -> Peer 13f7fb477c3849f59b5dc83f46b6e841 (127.2.169.253:32963): Couldn't send request to peer 13f7fb477c3849f59b5dc83f46b6e841. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.253:32963: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260807 07:53:01.303649 9292 consensus_queue.cc:579] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [LEADER]: Leader has been unable to successfully communicate with peer 13f7fb477c3849f59b5dc83f46b6e841 for more than 4 seconds (4.037s)
I20260807 07:53:01.304419 9292 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } peers { permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 32963 } } }
I20260807 07:53:01.304697 9292 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
W20260807 07:53:01.864248 9299 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:01.864681 9299 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260807 07:53:01.910609 8883 catalog_manager.cc:7202] Initiating ChangeConfig request to remove master 127.2.169.253:32963: tablet_id: "00000000000000000000000000000000"
type: REMOVE_PEER
server {
permanent_uuid: "13f7fb477c3849f59b5dc83f46b6e841"
}
dest_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
cas_config_opid_index: -1
I20260807 07:53:01.911904 8883 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } }
I20260807 07:53:01.913421 9300 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index -1 to 9, VOTER 13f7fb477c3849f59b5dc83f46b6e841 (127.2.169.253) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } }
I20260807 07:53:01.916433 9300 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to remove master 127.2.169.253:32963
I20260807 07:53:01.917016 9315 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } }
I20260807 07:53:01.917207 9315 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:01.917424 9299 tool_action_master.cc:569] Successfully removed master 127.2.169.253:32963 from the cluster. Please follow the next steps from the Kudu administration documentation on "Removing Kudu Masters from a Multi-Master Deployment" or "Recovering from a dead Kudu Master in a Multi-Master Deployment" as appropriate.
I20260807 07:53:02.106842 9145 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
b5117ca209af493b8f88ebb7d0b7f9d2 | 127.2.169.254:37689 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:35921 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
186e245ca80c433fb555c492e2050e90 | 127.2.169.193:37499 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.2.169.193 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:35921 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 07:53:02.147603 2727 log_verifier.cc:126] Checking tablet 355f713713c14c9bbc4febb329e0a746
I20260807 07:53:02.160516 2727 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 355f713713c14c9bbc4febb329e0a746
E20260807 07:53:02.597219 8883 master_service.cc:350] Failed removing master 127.2.169.253:32963. Not found: Master 127.2.169.253:32963 not found
I20260807 07:53:02.688635 2727 dynamic_multi_master-test.cc:992] Transferring leadership to master: 13f7fb477c3849f59b5dc83f46b6e841
I20260807 07:53:02.692999 8895 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
mode: GRACEFUL
new_leader_uuid: "13f7fb477c3849f59b5dc83f46b6e841"
from {username='slave'} at 127.0.0.1:49190
I20260807 07:53:02.693296 8895 raft_consensus.cc:606] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 LEADER]: Received request to transfer leadership to TS 13f7fb477c3849f59b5dc83f46b6e841
I20260807 07:53:02.693485 8895 raft_consensus.cc:630] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 LEADER]: Rejecting request to transfer leadership because tablet server 13f7fb477c3849f59b5dc83f46b6e841 is not a voter in the active config
I20260807 07:53:02.694607 2727 dynamic_multi_master-test.cc:999] Shutting down the old cluster
I20260807 07:53:02.694796 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9063
I20260807 07:53:02.701517 9219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:02.818238 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9063
I20260807 07:53:02.851248 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 8845
I20260807 07:53:02.852599 8916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:02.968609 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 8845
2026-08-07T07:53:02Z chronyd exiting
I20260807 07:53:03.015036 2727 dynamic_multi_master-test.cc:1002] Bringing up the migrated cluster
2026-08-07T07:53:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:53:03Z Disabled control of system clock
I20260807 07:53:03.063596 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:37689
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:37689
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260807 07:53:03.429767 9376 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:03.430079 9376 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:03.430163 9376 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:03.438429 9376 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:03.438529 9376 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:03.438584 9376 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:03.438632 9376 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:53:03.449524 9376 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:37689
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:37689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:03.451181 9376 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:03.453024 9376 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:03.461383 9382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:03.461433 9381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:03.462299 9384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:03.463096 9376 server_base.cc:1061] running on GCE node
I20260807 07:53:03.463762 9376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:03.465099 9376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:03.466296 9376 hybrid_clock.cc:648] HybridClock initialized: now 1786089183466230 us; error 53 us; skew 500 ppm
I20260807 07:53:03.466676 9376 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:03.468806 9376 webserver.cc:533] Webserver started at http://127.2.169.254:43185/ using document root <none> and password file <none>
I20260807 07:53:03.469655 9376 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:03.469808 9376 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:03.475184 9376 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 07:53:03.478101 9391 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:03.479449 9376 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:53:03.479629 9376 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "b5117ca209af493b8f88ebb7d0b7f9d2"
format_stamp: "Formatted at 2026-08-07 07:52:55 on dist-test-slave-bw88"
I20260807 07:53:03.480371 9376 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:03.499632 9376 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:03.500551 9376 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:03.500857 9376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:03.522951 9376 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:37689
I20260807 07:53:03.522959 9447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:37689 every 8 connection(s)
I20260807 07:53:03.524731 9376 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:53:03.530670 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9376
I20260807 07:53:03.530884 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:53:03.532266 9448 sys_catalog.cc:263] Verifying existing consensus state
I20260807 07:53:03.537798 9448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Bootstrap starting.
I20260807 07:53:03.558255 9448 log.cc:826] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:03.571318 9448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:53:03.571854 9448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Bootstrap complete.
I20260807 07:53:03.578807 9448 raft_consensus.cc:359] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } }
I20260807 07:53:03.579504 9448 raft_consensus.cc:740] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5117ca209af493b8f88ebb7d0b7f9d2, State: Initialized, Role: FOLLOWER
I20260807 07:53:03.580127 9448 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } }
I20260807 07:53:03.580340 9448 raft_consensus.cc:399] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:03.580431 9448 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:03.580580 9448 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 07:53:03.584160 9448 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } }
I20260807 07:53:03.584622 9448 leader_election.cc:304] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5117ca209af493b8f88ebb7d0b7f9d2; no voters:
I20260807 07:53:03.585222 9448 leader_election.cc:290] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 07:53:03.585536 9454 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 07:53:03.587173 9454 raft_consensus.cc:697] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [term 3 LEADER]: Becoming Leader. State: Replica: b5117ca209af493b8f88ebb7d0b7f9d2, State: Running, Role: LEADER
I20260807 07:53:03.587670 9448 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:53:03.587745 9454 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } }
I20260807 07:53:03.590737 9456 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5117ca209af493b8f88ebb7d0b7f9d2. Latest consensus state: current_term: 3 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } }
I20260807 07:53:03.590714 9455 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b5117ca209af493b8f88ebb7d0b7f9d2" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 37689 } } }
I20260807 07:53:03.591216 9455 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:03.591490 9456 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:03.598922 9463 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:53:03.606575 9463 catalog_manager.cc:679] Loaded metadata for table first_table [id=6117cb77e8684811be5c57c2b5957025]
I20260807 07:53:03.609164 9463 tablet_loader.cc:96] loaded metadata for tablet 355f713713c14c9bbc4febb329e0a746 (table first_table [id=6117cb77e8684811be5c57c2b5957025])
I20260807 07:53:03.609560 9463 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:53:03.611109 9463 catalog_manager.cc:1295] Loaded cluster ID: be2a12a592854eb5a47023448481ede9
I20260807 07:53:03.611229 9463 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:53:03.614338 9463 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:53:03.615628 9463 catalog_manager.cc:6081] T 00000000000000000000000000000000 P b5117ca209af493b8f88ebb7d0b7f9d2: Loaded TSK: 0
I20260807 07:53:03.616925 9463 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:53:03.629873 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:37689
--builtin_ntp_servers=127.2.169.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260807 07:53:04.000178 9475 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:04.000491 9475 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:04.000622 9475 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:04.009030 9475 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:04.009233 9475 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:53:04.018999 9475 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:37689
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:04.020732 9475 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:04.022691 9475 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:04.032097 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:04.032130 9482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:04.032915 9485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:04.033568 9475 server_base.cc:1061] running on GCE node
I20260807 07:53:04.034205 9475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:04.035002 9475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:04.036235 9475 hybrid_clock.cc:648] HybridClock initialized: now 1786089184036185 us; error 24 us; skew 500 ppm
I20260807 07:53:04.036536 9475 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:04.038723 9475 webserver.cc:533] Webserver started at http://127.2.169.193:36951/ using document root <none> and password file <none>
I20260807 07:53:04.039456 9475 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:04.039615 9475 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:04.045090 9475 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 07:53:04.047977 9493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:04.049244 9475 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 07:53:04.049443 9475 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "186e245ca80c433fb555c492e2050e90"
format_stamp: "Formatted at 2026-08-07 07:52:56 on dist-test-slave-bw88"
I20260807 07:53:04.050165 9475 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:04.085390 9475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:04.086664 9475 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:04.087037 9475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:04.088375 9475 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:53:04.091045 9501 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20260807 07:53:04.100087 9475 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 07:53:04.100220 9475 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260807 07:53:04.100370 9475 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260807 07:53:04.105412 9475 ts_tablet_manager.cc:616] Registered 1 tablets
I20260807 07:53:04.105543 9475 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s user 0.005s sys 0.000s
I20260807 07:53:04.106020 9501 tablet_bootstrap.cc:492] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Bootstrap starting.
I20260807 07:53:04.128625 9501 log.cc:826] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:04.140648 9501 tablet_bootstrap.cc:492] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 07:53:04.141672 9501 tablet_bootstrap.cc:492] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Bootstrap complete.
I20260807 07:53:04.143936 9501 ts_tablet_manager.cc:1403] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Time spent bootstrapping tablet: real 0.038s user 0.023s sys 0.013s
I20260807 07:53:04.147244 9475 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:44199
I20260807 07:53:04.147315 9624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:44199 every 8 connection(s)
I20260807 07:53:04.149144 9475 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:53:04.152786 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9475
I20260807 07:53:04.153057 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:53:04.157552 9501 raft_consensus.cc:359] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:53:04.158595 9501 raft_consensus.cc:740] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186e245ca80c433fb555c492e2050e90, State: Initialized, Role: FOLLOWER
I20260807 07:53:04.159533 9501 consensus_queue.cc:260] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:53:04.159793 9501 raft_consensus.cc:399] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:04.159935 9501 raft_consensus.cc:493] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:04.160117 9501 raft_consensus.cc:3060] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 1 FOLLOWER]: Advancing to term 2
I20260807 07:53:04.166533 9501 raft_consensus.cc:515] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:53:04.167176 9501 leader_election.cc:304] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186e245ca80c433fb555c492e2050e90; no voters:
I20260807 07:53:04.168000 9501 leader_election.cc:290] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 07:53:04.168131 9633 raft_consensus.cc:2804] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 07:53:04.169453 9625 heartbeater.cc:344] Connected to a master server at 127.2.169.254:37689
I20260807 07:53:04.169894 9625 heartbeater.cc:461] Registering TS with master...
I20260807 07:53:04.170408 9633 raft_consensus.cc:697] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [term 2 LEADER]: Becoming Leader. State: Replica: 186e245ca80c433fb555c492e2050e90, State: Running, Role: LEADER
I20260807 07:53:04.171258 9625 heartbeater.cc:507] Master 127.2.169.254:37689 requested a full tablet report, sending...
I20260807 07:53:04.171201 9633 consensus_queue.cc:237] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } }
I20260807 07:53:04.178755 9501 ts_tablet_manager.cc:1434] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90: Time spent starting tablet: real 0.035s user 0.021s sys 0.005s
I20260807 07:53:04.183368 9407 ts_manager.cc:194] Registered new tserver with Master: 186e245ca80c433fb555c492e2050e90 (127.2.169.193:44199)
I20260807 07:53:04.185145 9407 catalog_manager.cc:5697] T 355f713713c14c9bbc4febb329e0a746 P 186e245ca80c433fb555c492e2050e90 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "186e245ca80c433fb555c492e2050e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e245ca80c433fb555c492e2050e90" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 37499 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:04.196417 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:53:04.199079 9407 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:47929
I20260807 07:53:04.203188 9625 heartbeater.cc:499] Master 127.2.169.254:37689 was elected leader, sending a full tablet report...
I20260807 07:53:04.297974 9551 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
b5117ca209af493b8f88ebb7d0b7f9d2 | 127.2.169.254:37689 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:35275 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
186e245ca80c433fb555c492e2050e90 | 127.2.169.193:44199 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.2.169.193 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRemoveMasterTest.TestRemoveMaster_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.2.169.212:35275 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260807 07:53:04.334939 2727 log_verifier.cc:126] Checking tablet 355f713713c14c9bbc4febb329e0a746
I20260807 07:53:04.344663 2727 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 355f713713c14c9bbc4febb329e0a746
I20260807 07:53:04.374447 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9475
I20260807 07:53:04.381300 9618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:04.474148 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9475
I20260807 07:53:04.499097 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9376
I20260807 07:53:04.500375 9443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:04.611545 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9376
2026-08-07T07:53:04Z chronyd exiting
[ OK ] ParameterizedRemoveMasterTest.TestRemoveMaster/1 (10039 ms)
[----------] 1 test from ParameterizedRemoveMasterTest (10039 ms total)
[----------] 1 test from ParameterizedRecoverMasterTest
[ RUN ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/1
2026-08-07T07:53:04Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:53:04Z Disabled control of system clock
I20260807 07:53:04.707263 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:42545
--webserver_interface=127.2.169.254
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260807 07:53:05.064332 9680 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:05.064659 9680 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:05.064742 9680 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:05.073040 9680 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:05.073136 9680 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:05.073194 9680 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:05.073244 9680 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260807 07:53:05.073350 9680 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=0
I20260807 07:53:05.083505 9680 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:42545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:05.085589 9680 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:05.088379 9680 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:05.098934 9685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:05.098863 9686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:05.099516 9689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:05.100670 9680 server_base.cc:1061] running on GCE node
I20260807 07:53:05.101503 9680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:05.102969 9680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:05.104206 9680 hybrid_clock.cc:648] HybridClock initialized: now 1786089185104145 us; error 25 us; skew 500 ppm
I20260807 07:53:05.104561 9680 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:05.106791 9680 webserver.cc:533] Webserver started at http://127.2.169.254:40775/ using document root <none> and password file <none>
I20260807 07:53:05.107542 9680 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:05.107679 9680 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:05.108129 9680 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:05.110648 9680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "85836fd00d0b4e7f9dd80c844ee37519"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.111617 9680 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "85836fd00d0b4e7f9dd80c844ee37519"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.116994 9680 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 07:53:05.120507 9696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:05.121920 9680 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:53:05.122130 9680 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "85836fd00d0b4e7f9dd80c844ee37519"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.122341 9680 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:05.135528 9680 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:05.136322 9680 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:05.136631 9680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:05.157383 9754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:42545 every 8 connection(s)
I20260807 07:53:05.157433 9680 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:42545
I20260807 07:53:05.158941 9680 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:53:05.163334 9756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:05.165094 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9680
I20260807 07:53:05.165324 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:53:05.166597 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45107
--webserver_interface=127.2.169.253
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260807 07:53:05.169296 9756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:05.186733 9756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:53:05.187914 9699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
W20260807 07:53:05.189613 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45107: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
I20260807 07:53:05.238981 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } attempt: 1
W20260807 07:53:05.241428 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45107: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
I20260807 07:53:05.309751 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } attempt: 2
W20260807 07:53:05.311945 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45107: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
I20260807 07:53:05.403223 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } attempt: 3
W20260807 07:53:05.405483 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45107: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
W20260807 07:53:05.531809 9760 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:05.532127 9760 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:05.532274 9760 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:05.540323 9760 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:05.540402 9760 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:05.540526 9760 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:05.540606 9760 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260807 07:53:05.540761 9760 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=0
I20260807 07:53:05.548753 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } attempt: 4
W20260807 07:53:05.551132 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:45107: Network error: Client connection negotiation failed: client connection to 127.2.169.253:45107: connect: Connection refused (error 111)
I20260807 07:53:05.551260 9760 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:45107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:05.552917 9760 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:05.554889 9760 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:05.563485 9771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:05.563596 9770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:05.564301 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:05.565060 9760 server_base.cc:1061] running on GCE node
I20260807 07:53:05.565897 9760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:05.567239 9760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:05.568460 9760 hybrid_clock.cc:648] HybridClock initialized: now 1786089185568364 us; error 59 us; skew 500 ppm
I20260807 07:53:05.568728 9760 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:05.571049 9760 webserver.cc:533] Webserver started at http://127.2.169.253:45117/ using document root <none> and password file <none>
I20260807 07:53:05.571806 9760 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:05.571916 9760 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:05.572242 9760 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:05.574678 9760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "b1f16687ee13426cb547143f657d8e4d"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.575421 9760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "b1f16687ee13426cb547143f657d8e4d"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.580916 9760 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 07:53:05.584553 9781 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:05.585983 9760 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:53:05.586190 9760 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "b1f16687ee13426cb547143f657d8e4d"
format_stamp: "Formatted at 2026-08-07 07:53:05 on dist-test-slave-bw88"
I20260807 07:53:05.586402 9760 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:05.605648 9760 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:05.606570 9760 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:05.606962 9760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:05.629124 9760 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:45107
I20260807 07:53:05.629189 9846 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:45107 every 8 connection(s)
I20260807 07:53:05.630810 9760 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:53:05.635357 9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:05.640833 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9760
I20260807 07:53:05.641049 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:53:05.641582 9848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:05.642598 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:40495
--webserver_interface=127.2.169.252
--webserver_port=0
--builtin_ntp_servers=127.2.169.212:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260807 07:53:05.657152 9848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:05.666630 9848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:53:05.667752 9785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
W20260807 07:53:05.669533 9848 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.718894 9848 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 1
W20260807 07:53:05.721220 9848 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.789552 9848 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 2
W20260807 07:53:05.791767 9848 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.850507 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } attempt: 5
I20260807 07:53:05.857924 9756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:53:05.860437 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.883054 9848 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 3
W20260807 07:53:05.885231 9848 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.911681 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 1
W20260807 07:53:05.913888 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:05.982133 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 2
W20260807 07:53:05.984301 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
W20260807 07:53:06.003454 9851 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:06.003821 9851 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:06.003970 9851 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:06.012239 9851 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:06.012322 9851 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:06.012450 9851 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:06.012532 9851 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260807 07:53:06.012707 9851 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=0
I20260807 07:53:06.022686 9851 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.252:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.169.252
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:06.024364 9851 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:06.026247 9851 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260807 07:53:06.028592 9848 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 4
W20260807 07:53:06.030872 9848 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
W20260807 07:53:06.034610 9859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:06.034667 9858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:06.035720 9861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:06.036701 9851 server_base.cc:1061] running on GCE node
I20260807 07:53:06.037559 9851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:06.038865 9851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:06.040091 9851 hybrid_clock.cc:648] HybridClock initialized: now 1786089186040027 us; error 29 us; skew 500 ppm
I20260807 07:53:06.040439 9851 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:06.042582 9851 webserver.cc:533] Webserver started at http://127.2.169.252:37643/ using document root <none> and password file <none>
I20260807 07:53:06.043336 9851 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:06.043473 9851 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:06.043846 9851 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:06.046336 9851 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data/instance:
uuid: "e7af352b23b544d495c321f8c84767c6"
format_stamp: "Formatted at 2026-08-07 07:53:06 on dist-test-slave-bw88"
I20260807 07:53:06.047145 9851 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal/instance:
uuid: "e7af352b23b544d495c321f8c84767c6"
format_stamp: "Formatted at 2026-08-07 07:53:06 on dist-test-slave-bw88"
I20260807 07:53:06.052474 9851 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 07:53:06.055895 9868 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:06.057441 9851 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:53:06.057665 9851 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal
uuid: "e7af352b23b544d495c321f8c84767c6"
format_stamp: "Formatted at 2026-08-07 07:53:06 on dist-test-slave-bw88"
I20260807 07:53:06.057904 9851 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:06.078423 9851 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:06.079314 9851 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:06.079631 9851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:06.090545 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 3
W20260807 07:53:06.093506 9756 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.252:40495: Network error: Client connection negotiation failed: client connection to 127.2.169.252:40495: connect: Connection refused (error 111)
I20260807 07:53:06.108639 9851 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.252:40495
I20260807 07:53:06.108688 9938 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.252:40495 every 8 connection(s)
I20260807 07:53:06.110316 9851 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/data/info.pb
I20260807 07:53:06.114642 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 9851
I20260807 07:53:06.114888 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/master-2/wal/instance
I20260807 07:53:06.115332 9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:06.123910 9939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:06.136633 9939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:06.142117 9939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:06.151619 9939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Bootstrap starting.
I20260807 07:53:06.154846 9939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:06.155948 9939 log.cc:826] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:06.158993 9939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: No bootstrap required, opened a new log
I20260807 07:53:06.166469 9939 raft_consensus.cc:359] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.166788 9939 raft_consensus.cc:385] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:06.166901 9939 raft_consensus.cc:740] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7af352b23b544d495c321f8c84767c6, State: Initialized, Role: FOLLOWER
I20260807 07:53:06.167528 9939 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.168460 9947 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.168774 9947 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.169662 9939 sys_catalog.cc:565] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 07:53:06.178680 9961 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:53:06.178828 9961 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:53:06.246791 9947 raft_consensus.cc:493] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:53:06.247062 9947 raft_consensus.cc:515] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.248688 9947 leader_election.cc:290] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85836fd00d0b4e7f9dd80c844ee37519 (127.2.169.254:42545), b1f16687ee13426cb547143f657d8e4d (127.2.169.253:45107)
I20260807 07:53:06.270707 9756 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 4
I20260807 07:53:06.283479 9756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: Bootstrap starting.
I20260807 07:53:06.287889 9756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:06.289389 9756 log.cc:826] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:06.294411 9756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: No bootstrap required, opened a new log
I20260807 07:53:06.302249 9756 raft_consensus.cc:359] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.302640 9756 raft_consensus.cc:385] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:06.302784 9756 raft_consensus.cc:740] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85836fd00d0b4e7f9dd80c844ee37519, State: Initialized, Role: FOLLOWER
I20260807 07:53:06.303500 9756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.305130 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.305869 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.306299 9756 sys_catalog.cc:565] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:53:06.306354 9725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7af352b23b544d495c321f8c84767c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85836fd00d0b4e7f9dd80c844ee37519" is_pre_election: true
I20260807 07:53:06.306913 9725 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7af352b23b544d495c321f8c84767c6 in term 0.
I20260807 07:53:06.307718 9872 leader_election.cc:304] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85836fd00d0b4e7f9dd80c844ee37519, e7af352b23b544d495c321f8c84767c6; no voters:
I20260807 07:53:06.308274 9947 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 07:53:06.308431 9947 raft_consensus.cc:493] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 07:53:06.308557 9947 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:06.312083 9947 raft_consensus.cc:515] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.312884 9947 leader_election.cc:290] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [CANDIDATE]: Term 1 election: Requested vote from peers 85836fd00d0b4e7f9dd80c844ee37519 (127.2.169.254:42545), b1f16687ee13426cb547143f657d8e4d (127.2.169.253:45107)
I20260807 07:53:06.313899 9725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7af352b23b544d495c321f8c84767c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85836fd00d0b4e7f9dd80c844ee37519"
I20260807 07:53:06.314096 9725 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:06.319751 9725 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7af352b23b544d495c321f8c84767c6 in term 1.
I20260807 07:53:06.320238 9872 leader_election.cc:304] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85836fd00d0b4e7f9dd80c844ee37519, e7af352b23b544d495c321f8c84767c6; no voters:
I20260807 07:53:06.320565 9947 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:06.321575 9947 raft_consensus.cc:697] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [term 1 LEADER]: Becoming Leader. State: Replica: e7af352b23b544d495c321f8c84767c6, State: Running, Role: LEADER
I20260807 07:53:06.322081 9947 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.327255 9973 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7af352b23b544d495c321f8c84767c6. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.327664 9973 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:06.330106 9848 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } attempt: 5
W20260807 07:53:06.331878 9875 tablet.cc:2406] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:53:06.335565 9983 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 07:53:06.338209 9983 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 07:53:06.348791 9848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: Bootstrap starting.
I20260807 07:53:06.352851 9725 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [term 1 FOLLOWER]: Refusing update from remote peer e7af352b23b544d495c321f8c84767c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:53:06.353204 9848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:06.354091 9973 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 07:53:06.355715 9848 log.cc:826] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:06.372751 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7af352b23b544d495c321f8c84767c6. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.373018 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.376355 9848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: No bootstrap required, opened a new log
W20260807 07:53:06.378126 9986 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 07:53:06.378350 9986 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 07:53:06.384608 9848 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.384979 9848 raft_consensus.cc:385] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:06.385103 9848 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f16687ee13426cb547143f657d8e4d, State: Initialized, Role: FOLLOWER
I20260807 07:53:06.384986 9947 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.385269 9947 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:06.385946 9848 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } }
I20260807 07:53:06.388525 9983 catalog_manager.cc:1383] Generated new cluster ID: c8c25d9484164ca6b1c42aa3f43af76f
I20260807 07:53:06.388655 9983 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 07:53:06.396322 9995 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.396680 9995 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.399144 9848 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 07:53:06.398880 9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7af352b23b544d495c321f8c84767c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f16687ee13426cb547143f657d8e4d"
I20260807 07:53:06.398854 9814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7af352b23b544d495c321f8c84767c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1f16687ee13426cb547143f657d8e4d" is_pre_election: true
I20260807 07:53:06.399252 9812 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:06.403138 9812 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [term 1 FOLLOWER]: Refusing update from remote peer e7af352b23b544d495c321f8c84767c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 07:53:06.404065 9973 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 07:53:06.409982 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.410276 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.414744 9983 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 07:53:06.416839 9983 catalog_manager.cc:1540] Loading token signing keys...
I20260807 07:53:06.448366 9983 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Generated new TSK 0
I20260807 07:53:06.449605 9983 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 07:53:06.462409 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.169.193:0
--local_ip_for_outbound_sockets=127.2.169.193
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--builtin_ntp_servers=127.2.169.212:44137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260807 07:53:06.470865 10003 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.471280 10003 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.471745 9995 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7af352b23b544d495c321f8c84767c6. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.471958 9995 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 07:53:06.473430 9973 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:06.473655 9973 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:06.486718 10015 catalog_manager.cc:1295] Loaded cluster ID: c8c25d9484164ca6b1c42aa3f43af76f
I20260807 07:53:06.486819 10015 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: loading cluster ID for follower catalog manager: success
I20260807 07:53:06.490829 10015 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: acquiring CA information for follower catalog manager: success
I20260807 07:53:06.493425 10015 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b1f16687ee13426cb547143f657d8e4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:53:06.583302 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000115
I20260807 07:53:06.610705 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000059
I20260807 07:53:06.640806 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000093
I20260807 07:53:06.708532 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.124s user 0.024s sys 0.035s Metrics: {"bytes_written":6027,"cfile_init":1,"dirs.queue_time_us":9947,"dirs.run_cpu_time_us":291,"dirs.run_wall_time_us":1724,"drs_written":1,"lbm_read_time_us":128,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1333,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":10121,"threads_started":1}
I20260807 07:53:06.722298 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260807 07:53:06.723670 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260807 07:53:06.734912 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.123s user 0.031s sys 0.016s Metrics: {"bytes_written":6027,"cfile_init":1,"dirs.queue_time_us":16326,"dirs.run_cpu_time_us":203,"dirs.run_wall_time_us":1999,"drs_written":1,"lbm_read_time_us":134,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1330,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":11346,"threads_started":1}
I20260807 07:53:06.736400 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260807 07:53:06.748186 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":117,"lbm_reads_lt_1ms":4}
I20260807 07:53:06.751096 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.109s user 0.029s sys 0.017s Metrics: {"bytes_written":6027,"cfile_init":1,"dirs.queue_time_us":6487,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1898,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1160,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":10401,"threads_started":1}
I20260807 07:53:06.752483 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260807 07:53:06.753350 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
W20260807 07:53:07.084136 10009 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:07.084607 10009 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:07.084748 10009 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:07.099159 10009 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:07.099406 10009 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.169.193
I20260807 07:53:07.113082 10009 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:44137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.169.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.169.193
--webserver_port=0
--tserver_master_addrs=127.2.169.254:42545,127.2.169.253:45107,127.2.169.252:40495
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.169.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
I20260807 07:53:07.115283 10009 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:07.117733 10009 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:07.129395 10026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:07.130976 10027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:07.138438 10030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:07.138845 10009 server_base.cc:1061] running on GCE node
I20260807 07:53:07.139480 10009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:07.140575 10009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:07.141804 10009 hybrid_clock.cc:648] HybridClock initialized: now 1786089187141746 us; error 50 us; skew 500 ppm
I20260807 07:53:07.142153 10009 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 07:53:07.144498 10009 webserver.cc:533] Webserver started at http://127.2.169.193:38097/ using document root <none> and password file <none>
I20260807 07:53:07.145380 10009 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:07.145529 10009 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:07.145925 10009 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:07.148411 10009 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data/instance:
uuid: "9ecc5f2dc8874ab884abb1a8392e4518"
format_stamp: "Formatted at 2026-08-07 07:53:07 on dist-test-slave-bw88"
I20260807 07:53:07.149222 10009 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance:
uuid: "9ecc5f2dc8874ab884abb1a8392e4518"
format_stamp: "Formatted at 2026-08-07 07:53:07 on dist-test-slave-bw88"
I20260807 07:53:07.154964 10009 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:07.158609 10037 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:07.160108 10009 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 07:53:07.160310 10009 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
uuid: "9ecc5f2dc8874ab884abb1a8392e4518"
format_stamp: "Formatted at 2026-08-07 07:53:07 on dist-test-slave-bw88"
I20260807 07:53:07.160531 10009 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:07.190981 10009 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:07.192044 10009 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:07.192375 10009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:07.193467 10009 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 07:53:07.195262 10009 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 07:53:07.195396 10009 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:07.195530 10009 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 07:53:07.195621 10009 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:07.232321 10009 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.193:45819
I20260807 07:53:07.232414 10176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.193:45819 every 8 connection(s)
I20260807 07:53:07.233916 10009 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/data/info.pb
I20260807 07:53:07.237643 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 10009
I20260807 07:53:07.238039 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/instance
I20260807 07:53:07.257721 10177 heartbeater.cc:344] Connected to a master server at 127.2.169.252:40495
I20260807 07:53:07.257856 10178 heartbeater.cc:344] Connected to a master server at 127.2.169.254:42545
I20260807 07:53:07.257941 10179 heartbeater.cc:344] Connected to a master server at 127.2.169.253:45107
I20260807 07:53:07.258242 10179 heartbeater.cc:461] Registering TS with master...
I20260807 07:53:07.258235 10178 heartbeater.cc:461] Registering TS with master...
I20260807 07:53:07.258232 10177 heartbeater.cc:461] Registering TS with master...
I20260807 07:53:07.259557 10178 heartbeater.cc:507] Master 127.2.169.254:42545 requested a full tablet report, sending...
I20260807 07:53:07.259588 10177 heartbeater.cc:507] Master 127.2.169.252:40495 requested a full tablet report, sending...
I20260807 07:53:07.259531 10179 heartbeater.cc:507] Master 127.2.169.253:45107 requested a full tablet report, sending...
I20260807 07:53:07.262420 9802 ts_manager.cc:194] Registered new tserver with Master: 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193:45819)
I20260807 07:53:07.262439 9887 ts_manager.cc:194] Registered new tserver with Master: 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193:45819)
I20260807 07:53:07.262486 9712 ts_manager.cc:194] Registered new tserver with Master: 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193:45819)
I20260807 07:53:07.264619 9887 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.2.169.193:40915
I20260807 07:53:07.268319 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:53:07.276069 2727 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260807 07:53:07.326108 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36008:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.359333 10099 tablet_service.cc:1511] Processing CreateTablet for tablet dd3b9f8b5c2e4263a8285cf5ec6c0efa (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.1 [id=92a98fca64da4fe88b20a2052aeb1efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.360747 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3b9f8b5c2e4263a8285cf5ec6c0efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.370884 10215 tablet_bootstrap.cc:492] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.374003 10215 tablet_bootstrap.cc:654] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.375155 10215 log.cc:826] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:07.378243 10215 tablet_bootstrap.cc:492] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.378515 10215 ts_tablet_manager.cc:1403] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 07:53:07.383571 10215 raft_consensus.cc:359] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.383903 10215 raft_consensus.cc:385] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.384019 10215 raft_consensus.cc:740] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.384727 10215 consensus_queue.cc:260] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.385004 10215 raft_consensus.cc:399] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.385169 10215 raft_consensus.cc:493] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.385396 10215 raft_consensus.cc:3060] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.387303 9986 catalog_manager.cc:1295] Loaded cluster ID: c8c25d9484164ca6b1c42aa3f43af76f
I20260807 07:53:07.387455 9986 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: loading cluster ID for follower catalog manager: success
I20260807 07:53:07.387601 10215 raft_consensus.cc:515] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.388047 10215 leader_election.cc:304] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.388737 10215 leader_election.cc:290] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.388938 10218 raft_consensus.cc:2804] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.390391 10177 heartbeater.cc:499] Master 127.2.169.252:40495 was elected leader, sending a full tablet report...
I20260807 07:53:07.391115 10218 raft_consensus.cc:697] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.391535 10215 ts_tablet_manager.cc:1434] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 07:53:07.391753 10218 consensus_queue.cc:237] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.393019 9986 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: acquiring CA information for follower catalog manager: success
I20260807 07:53:07.396799 9986 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 85836fd00d0b4e7f9dd80c844ee37519: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 07:53:07.399446 9886 catalog_manager.cc:5697] T dd3b9f8b5c2e4263a8285cf5ec6c0efa P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.438194 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36018:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.450814 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 58737d957c28467798c9627a74687818 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.2 [id=ec8b0aa3324d4faea88b76e8438d2513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.451428 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58737d957c28467798c9627a74687818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.459290 10215 tablet_bootstrap.cc:492] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.461051 10215 tablet_bootstrap.cc:654] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.463573 10215 tablet_bootstrap.cc:492] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.463763 10215 ts_tablet_manager.cc:1403] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:07.464841 10215 raft_consensus.cc:359] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.465026 10215 raft_consensus.cc:385] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.465152 10215 raft_consensus.cc:740] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.465451 10215 consensus_queue.cc:260] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.465674 10215 raft_consensus.cc:399] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.465751 10215 raft_consensus.cc:493] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.465833 10215 raft_consensus.cc:3060] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.467784 10215 raft_consensus.cc:515] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.468039 10215 leader_election.cc:304] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.468246 10215 leader_election.cc:290] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.468338 10218 raft_consensus.cc:2804] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.468597 10218 raft_consensus.cc:697] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.468832 10215 ts_tablet_manager.cc:1434] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 07:53:07.468899 10218 consensus_queue.cc:237] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.471798 9886 catalog_manager.cc:5697] T 58737d957c28467798c9627a74687818 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.476290 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000213
I20260807 07:53:07.490236 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.004s sys 0.007s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":1797,"dirs.run_cpu_time_us":289,"dirs.run_wall_time_us":1685,"drs_written":1,"lbm_read_time_us":46,"lbm_reads_lt_1ms":4,"lbm_write_time_us":761,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"thread_start_us":1947,"threads_started":1}
I20260807 07:53:07.491794 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260807 07:53:07.492516 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.493216 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260807 07:53:07.500232 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000213
I20260807 07:53:07.505200 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000209
I20260807 07:53:07.517064 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36028:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.525177 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.022s sys 0.005s Metrics: {"bytes_written":6920,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2276,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3478,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":750,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1195,"drs_written":1,"lbm_read_time_us":1270,"lbm_reads_lt_1ms":56,"lbm_write_time_us":982,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":7}
I20260807 07:53:07.526086 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 477 bytes on disk
I20260807 07:53:07.526697 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.530237 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:07.545692 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 5499395d122d429bb9b698a10a54c83e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.3 [id=e6a7ce4ac3944156a71492b492ddde34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.546239 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5499395d122d429bb9b698a10a54c83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.553521 10215 tablet_bootstrap.cc:492] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.555437 10215 tablet_bootstrap.cc:654] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.556584 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.006s sys 0.008s Metrics: {"bytes_written":5043,"cfile_init":1,"dirs.queue_time_us":211,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1254,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1055,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:07.557878 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.331131
I20260807 07:53:07.559113 10215 tablet_bootstrap.cc:492] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.559329 10215 ts_tablet_manager.cc:1403] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:07.560114 10215 raft_consensus.cc:359] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.560333 10215 raft_consensus.cc:385] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.560417 10215 raft_consensus.cc:740] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.560695 10215 consensus_queue.cc:260] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.560873 10215 raft_consensus.cc:399] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.560981 10215 raft_consensus.cc:493] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.561101 10215 raft_consensus.cc:3060] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.562595 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.013s sys 0.006s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":31044,"dirs.run_cpu_time_us":273,"dirs.run_wall_time_us":3827,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1069,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"thread_start_us":31156,"threads_started":1}
I20260807 07:53:07.563297 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.013s sys 0.007s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":38917,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":3291,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":954,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"thread_start_us":39407,"threads_started":1}
I20260807 07:53:07.564090 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260807 07:53:07.564142 10215 raft_consensus.cc:515] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.564373 10215 leader_election.cc:304] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.564689 10218 raft_consensus.cc:2804] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.564739 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 440 bytes on disk
I20260807 07:53:07.564852 10218 raft_consensus.cc:697] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.565105 10218 consensus_queue.cc:237] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.565447 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.566104 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260807 07:53:07.567271 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":2359,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260807 07:53:07.568081 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260807 07:53:07.571239 10215 leader_election.cc:290] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.572109 10215 ts_tablet_manager.cc:1434] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.001s sys 0.004s
I20260807 07:53:07.586513 9886 catalog_manager.cc:5697] T 5499395d122d429bb9b698a10a54c83e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.593698 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.020s sys 0.012s Metrics: {"bytes_written":7348,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2496,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3736,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":785,"dirs.run_cpu_time_us":775,"dirs.run_wall_time_us":4076,"drs_written":1,"lbm_read_time_us":1193,"lbm_reads_lt_1ms":60,"lbm_write_time_us":996,"lbm_writes_lt_1ms":27,"mutex_wait_us":161,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":9,"thread_start_us":10124,"threads_started":1}
I20260807 07:53:07.594754 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 546 bytes on disk
I20260807 07:53:07.595598 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.600778 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:07.608302 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.001s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:07.609217 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121014
I20260807 07:53:07.626049 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.060s user 0.028s sys 0.030s Metrics: {"bytes_written":6920,"cfile_cache_hit":23,"cfile_cache_hit_bytes":3891,"cfile_cache_miss":17,"cfile_cache_miss_bytes":1863,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":3311,"dirs.run_cpu_time_us":866,"dirs.run_wall_time_us":4352,"drs_written":1,"lbm_read_time_us":993,"lbm_reads_lt_1ms":33,"lbm_write_time_us":1041,"lbm_writes_lt_1ms":27,"mutex_wait_us":258,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":7,"thread_start_us":32292,"threads_started":2}
I20260807 07:53:07.627100 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 477 bytes on disk
I20260807 07:53:07.626924 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.008s sys 0.008s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2428,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":84,"lbm_read_time_us":573,"lbm_reads_lt_1ms":21,"lbm_write_time_us":454,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:07.627724 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.627863 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:07.628263 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000018
I20260807 07:53:07.633162 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":234,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:07.633807 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.120616
I20260807 07:53:07.637120 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.069s user 0.021s sys 0.030s Metrics: {"bytes_written":6920,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2276,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3478,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1254,"dirs.run_cpu_time_us":798,"dirs.run_wall_time_us":3837,"drs_written":1,"lbm_read_time_us":3001,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":1041,"lbm_writes_lt_1ms":27,"mutex_wait_us":259,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":7,"thread_start_us":32000,"threads_started":2}
I20260807 07:53:07.638321 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 477 bytes on disk
I20260807 07:53:07.639045 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.639703 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000021
I20260807 07:53:07.644452 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.009s sys 0.000s Metrics: {"bytes_written":5194,"cfile_init":1,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":810,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:07.644625 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2465,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":82,"lbm_read_time_us":587,"lbm_reads_lt_1ms":25,"lbm_write_time_us":359,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:07.646214 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.393919
I20260807 07:53:07.653700 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.003s sys 0.010s Metrics: {"bytes_written":5194,"cfile_init":1,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":156,"dirs.run_wall_time_us":886,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":964,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:07.654964 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.393919
I20260807 07:53:07.696050 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.022s sys 0.013s Metrics: {"bytes_written":7385,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2648,"cfile_cache_miss":28,"cfile_cache_miss_bytes":4308,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":966,"dirs.run_cpu_time_us":989,"dirs.run_wall_time_us":4834,"drs_written":1,"lbm_read_time_us":1837,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1341,"lbm_writes_lt_1ms":27,"mutex_wait_us":140,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":9,"spinlock_wait_cycles":290816,"thread_start_us":7071,"threads_started":1}
I20260807 07:53:07.697152 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 580 bytes on disk
I20260807 07:53:07.697901 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.709767 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.025s sys 0.011s Metrics: {"bytes_written":7385,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2648,"cfile_cache_miss":28,"cfile_cache_miss_bytes":4308,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":5520,"dirs.run_cpu_time_us":697,"dirs.run_wall_time_us":3519,"drs_written":1,"lbm_read_time_us":4587,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":59,"lbm_write_time_us":1952,"lbm_writes_lt_1ms":27,"mutex_wait_us":2723,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":9,"thread_start_us":11489,"threads_started":1}
I20260807 07:53:07.710963 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 580 bytes on disk
I20260807 07:53:07.711746 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":113,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.713483 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36032:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.742949 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 22613385859a4e83921ddafcd3438bb5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.4 [id=e4b238e426b9412eb20ad06f30b67e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.743525 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22613385859a4e83921ddafcd3438bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.748600 10215 tablet_bootstrap.cc:492] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.750916 10215 tablet_bootstrap.cc:654] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.753860 10215 tablet_bootstrap.cc:492] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.754041 10215 ts_tablet_manager.cc:1403] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:07.755100 10215 raft_consensus.cc:359] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.755304 10215 raft_consensus.cc:385] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.755427 10215 raft_consensus.cc:740] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.755739 10215 consensus_queue.cc:260] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.755959 10215 raft_consensus.cc:399] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.756081 10215 raft_consensus.cc:493] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.756227 10215 raft_consensus.cc:3060] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.760144 10215 raft_consensus.cc:515] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.760432 10215 leader_election.cc:304] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.760676 10215 leader_election.cc:290] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.760780 10218 raft_consensus.cc:2804] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.760975 10218 raft_consensus.cc:697] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.761217 10218 consensus_queue.cc:237] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.761675 10215 ts_tablet_manager.cc:1434] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 07:53:07.764186 9886 catalog_manager.cc:5697] T 22613385859a4e83921ddafcd3438bb5 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.815544 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36044:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.830421 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 1a6b18469e9e46a788c010d46d330549 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.5 [id=2c32db8d8096436daf9541b3c7b97c32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.831058 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a6b18469e9e46a788c010d46d330549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.836858 10215 tablet_bootstrap.cc:492] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.839079 10215 tablet_bootstrap.cc:654] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.841991 10215 tablet_bootstrap.cc:492] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.842177 10215 ts_tablet_manager.cc:1403] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:07.843138 10215 raft_consensus.cc:359] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.843361 10215 raft_consensus.cc:385] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.843466 10215 raft_consensus.cc:740] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.843750 10215 consensus_queue.cc:260] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.843945 10215 raft_consensus.cc:399] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.844063 10215 raft_consensus.cc:493] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.844184 10215 raft_consensus.cc:3060] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.847962 10215 raft_consensus.cc:515] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.848326 10215 leader_election.cc:304] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.848759 10215 leader_election.cc:290] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.848805 10218 raft_consensus.cc:2804] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.849092 10218 raft_consensus.cc:697] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.849539 10218 consensus_queue.cc:237] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.851815 10215 ts_tablet_manager.cc:1434] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260807 07:53:07.853109 9886 catalog_manager.cc:5697] T 1a6b18469e9e46a788c010d46d330549 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.895817 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000094
I20260807 07:53:07.910094 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.012s sys 0.001s Metrics: {"bytes_written":5626,"cfile_init":1,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1429,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1104,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:07.910919 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 465 bytes on disk
I20260807 07:53:07.911561 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.912254 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.404049
I20260807 07:53:07.913388 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36046:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:07.931591 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 30053df3a63c4bb79b22e22d6015289e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.6 [id=8bf628f5b9244a9587fe593ca7d2d4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:07.932090 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30053df3a63c4bb79b22e22d6015289e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:07.937757 10215 tablet_bootstrap.cc:492] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:07.939203 10215 tablet_bootstrap.cc:654] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:07.948233 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.020s sys 0.012s Metrics: {"bytes_written":8259,"cfile_cache_hit":14,"cfile_cache_hit_bytes":3449,"cfile_cache_miss":34,"cfile_cache_miss_bytes":6249,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":2446,"dirs.run_cpu_time_us":999,"dirs.run_wall_time_us":5335,"drs_written":1,"lbm_read_time_us":1322,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1007,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":13,"thread_start_us":8142,"threads_started":2}
I20260807 07:53:07.948973 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:07.949374 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 763 bytes on disk
I20260807 07:53:07.950103 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.950717 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:07.950780 10215 tablet_bootstrap.cc:492] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:07.950945 10215 ts_tablet_manager.cc:1403] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260807 07:53:07.952036 10215 raft_consensus.cc:359] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.952261 10215 raft_consensus.cc:385] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:07.952356 10215 raft_consensus.cc:740] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:07.952638 10215 consensus_queue.cc:260] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.952852 10215 raft_consensus.cc:399] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:07.952936 10215 raft_consensus.cc:493] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:07.953083 10215 raft_consensus.cc:3060] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:07.958235 10215 raft_consensus.cc:515] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.958657 10215 leader_election.cc:304] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:07.959091 10218 raft_consensus.cc:2804] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:07.959273 10218 raft_consensus.cc:697] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:07.959544 10218 consensus_queue.cc:237] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:07.962759 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6029,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":940,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":988,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:07.962985 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260807 07:53:07.962924 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.010s sys 0.000s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":151,"dirs.run_wall_time_us":905,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1112,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:07.963619 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:07.963619 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.475043
I20260807 07:53:07.964246 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.964898 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.326495
I20260807 07:53:07.976557 9887 catalog_manager.cc:5697] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:07.982023 10215 leader_election.cc:290] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:07.982651 10215 ts_tablet_manager.cc:1434] T 30053df3a63c4bb79b22e22d6015289e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.032s user 0.006s sys 0.002s
I20260807 07:53:07.987824 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.012s sys 0.006s Metrics: {"bytes_written":8654,"cfile_cache_hit":12,"cfile_cache_hit_bytes":3721,"cfile_cache_miss":28,"cfile_cache_miss_bytes":5711,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":37,"dirs.run_cpu_time_us":128,"dirs.run_wall_time_us":764,"drs_written":1,"lbm_read_time_us":1184,"lbm_reads_lt_1ms":56,"lbm_write_time_us":961,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":15}
I20260807 07:53:07.988565 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 826 bytes on disk
I20260807 07:53:07.989473 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.990079 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:07.990131 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.010s sys 0.005s Metrics: {"bytes_written":6029,"cfile_init":1,"dirs.queue_time_us":58,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1008,"drs_written":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1094,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:07.991570 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 528 bytes on disk
I20260807 07:53:07.992314 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260807 07:53:07.993196 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.475043
I20260807 07:53:07.995050 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.002s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:08.014608 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.021s sys 0.000s Metrics: {"bytes_written":8654,"cfile_cache_hit":12,"cfile_cache_hit_bytes":3716,"cfile_cache_miss":28,"cfile_cache_miss_bytes":5922,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":790,"dirs.run_cpu_time_us":722,"dirs.run_wall_time_us":4387,"drs_written":1,"lbm_read_time_us":1469,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1954,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":15,"spinlock_wait_cycles":3840,"thread_start_us":561,"threads_started":1}
I20260807 07:53:08.015637 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:08.023269 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.001s sys 0.002s Metrics: {"bytes_written":390,"delete_count":0,"lbm_write_time_us":122,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:08.023819 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.100958
I20260807 07:53:08.033949 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.027s sys 0.008s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":3716,"cfile_cache_miss":28,"cfile_cache_miss_bytes":5922,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":441,"dirs.run_cpu_time_us":786,"dirs.run_wall_time_us":3808,"drs_written":1,"lbm_read_time_us":1472,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1293,"lbm_writes_lt_1ms":31,"mutex_wait_us":33,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":15,"spinlock_wait_cycles":6400,"thread_start_us":6552,"threads_started":1,"update_count":2}
I20260807 07:53:08.035231 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.100958
I20260807 07:53:08.038431 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":3383,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":102,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1006,"lbm_read_time_us":772,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1636,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"peak_mem_usage":32768,"reinsert_count":0,"update_count":2}
I20260807 07:53:08.039426 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 826 bytes on disk
I20260807 07:53:08.040112 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.049683 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.014s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":3383,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":175,"lbm_read_time_us":570,"lbm_reads_lt_1ms":25,"lbm_write_time_us":352,"lbm_writes_lt_1ms":9,"mutex_wait_us":31,"peak_mem_usage":32768,"reinsert_count":0,"update_count":2}
I20260807 07:53:08.050565 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 826 bytes on disk
I20260807 07:53:08.052505 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.062805 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.075392 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 156ce6617b6844fa91d186312a1db3a3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.7 [id=50bd157ce5d741419f93ce4434bae899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.075910 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 156ce6617b6844fa91d186312a1db3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.080821 10215 tablet_bootstrap.cc:492] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.082404 10215 tablet_bootstrap.cc:654] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.085094 10215 tablet_bootstrap.cc:492] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.085306 10215 ts_tablet_manager.cc:1403] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.086099 10215 raft_consensus.cc:359] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.086303 10215 raft_consensus.cc:385] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.086442 10215 raft_consensus.cc:740] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.086807 10215 consensus_queue.cc:260] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.086988 10215 raft_consensus.cc:399] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.087119 10215 raft_consensus.cc:493] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.087244 10215 raft_consensus.cc:3060] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.089607 10215 raft_consensus.cc:515] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.089866 10215 leader_election.cc:304] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.090132 10215 leader_election.cc:290] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.090224 10219 raft_consensus.cc:2804] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.090431 10219 raft_consensus.cc:697] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.090692 10215 ts_tablet_manager.cc:1434] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260807 07:53:08.090723 10219 consensus_queue.cc:237] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.093924 9887 catalog_manager.cc:5697] T 156ce6617b6844fa91d186312a1db3a3 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.142103 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36066:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.155054 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 8d6874171e784408b46649853d30e023 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.8 [id=676f2d648a0346c6a489cf601fc12f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.155532 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d6874171e784408b46649853d30e023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.161243 10215 tablet_bootstrap.cc:492] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.162760 10215 tablet_bootstrap.cc:654] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.165463 10215 tablet_bootstrap.cc:492] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.165620 10215 ts_tablet_manager.cc:1403] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.166397 10215 raft_consensus.cc:359] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.166594 10215 raft_consensus.cc:385] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.166707 10215 raft_consensus.cc:740] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.166966 10215 consensus_queue.cc:260] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.167137 10215 raft_consensus.cc:399] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.167237 10215 raft_consensus.cc:493] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.167353 10215 raft_consensus.cc:3060] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.169690 10215 raft_consensus.cc:515] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.169997 10215 leader_election.cc:304] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.170233 10215 leader_election.cc:290] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.170329 10219 raft_consensus.cc:2804] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.170508 10219 raft_consensus.cc:697] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.170787 10215 ts_tablet_manager.cc:1434] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 07:53:08.170720 10219 consensus_queue.cc:237] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.174249 9887 catalog_manager.cc:5697] T 8d6874171e784408b46649853d30e023 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.208168 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36082:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.219647 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0d0e465fe9c34c59869f7f6167717a9d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.9 [id=9e50a7c35d524c9b89aaa0390982cbf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.220211 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d0e465fe9c34c59869f7f6167717a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.225659 10215 tablet_bootstrap.cc:492] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.226920 10215 tablet_bootstrap.cc:654] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.229564 10215 tablet_bootstrap.cc:492] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.229739 10215 ts_tablet_manager.cc:1403] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 07:53:08.230327 10215 raft_consensus.cc:359] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.230509 10215 raft_consensus.cc:385] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.230625 10215 raft_consensus.cc:740] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.230876 10215 consensus_queue.cc:260] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.231035 10215 raft_consensus.cc:399] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.231148 10215 raft_consensus.cc:493] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.231263 10215 raft_consensus.cc:3060] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.234297 10215 raft_consensus.cc:515] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.234545 10215 leader_election.cc:304] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.234767 10215 leader_election.cc:290] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.234858 10219 raft_consensus.cc:2804] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.235034 10219 raft_consensus.cc:697] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.235206 10215 ts_tablet_manager.cc:1434] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 07:53:08.235285 10219 consensus_queue.cc:237] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.238897 9887 catalog_manager.cc:5697] T 0d0e465fe9c34c59869f7f6167717a9d P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.245864 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:08.256709 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.006s sys 0.001s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1351,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":755,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:08.257508 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 519 bytes on disk
I20260807 07:53:08.258016 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.258595 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.417756
I20260807 07:53:08.278293 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.015s sys 0.004s Metrics: {"bytes_written":10027,"cfile_cache_hit":13,"cfile_cache_hit_bytes":4853,"cfile_cache_miss":31,"cfile_cache_miss_bytes":7931,"cfile_init":7,"delta_iterators_relevant":6,"dirs.queue_time_us":593,"dirs.run_cpu_time_us":352,"dirs.run_wall_time_us":1770,"drs_written":1,"lbm_read_time_us":866,"lbm_reads_lt_1ms":59,"lbm_write_time_us":884,"lbm_writes_lt_1ms":27,"mutex_wait_us":132,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":21,"spinlock_wait_cycles":6400}
I20260807 07:53:08.279220 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1110 bytes on disk
I20260807 07:53:08.279811 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.291327 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000091
I20260807 07:53:08.297871 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36098:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.303956 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:08.307365 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.009s sys 0.003s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":86,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":1179,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1048,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:08.308424 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.415900
I20260807 07:53:08.314107 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a0563194f7fa4582ba2683a264ec6df2 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.10 [id=2858fd114f7346cabe45f3a78be33637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.314694 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0563194f7fa4582ba2683a264ec6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.316581 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":178,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":964,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":961,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:08.317451 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 519 bytes on disk
I20260807 07:53:08.318004 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.318614 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.415900
I20260807 07:53:08.320014 10215 tablet_bootstrap.cc:492] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.321889 10215 tablet_bootstrap.cc:654] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.324749 10215 tablet_bootstrap.cc:492] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.324910 10215 ts_tablet_manager.cc:1403] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:08.325780 10215 raft_consensus.cc:359] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.325943 10215 raft_consensus.cc:385] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.326032 10215 raft_consensus.cc:740] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.326340 10215 consensus_queue.cc:260] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.326514 10215 raft_consensus.cc:399] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.326601 10215 raft_consensus.cc:493] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.326711 10215 raft_consensus.cc:3060] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.329432 10215 raft_consensus.cc:515] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.329689 10215 leader_election.cc:304] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.329900 10215 leader_election.cc:290] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.330031 10219 raft_consensus.cc:2804] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.330230 10219 raft_consensus.cc:697] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.330483 10219 consensus_queue.cc:237] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.331373 10215 ts_tablet_manager.cc:1434] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:08.336292 9887 catalog_manager.cc:5697] T a0563194f7fa4582ba2683a264ec6df2 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.352492 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.022s sys 0.001s Metrics: {"bytes_written":10027,"cfile_cache_hit":13,"cfile_cache_hit_bytes":4890,"cfile_cache_miss":31,"cfile_cache_miss_bytes":8004,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":326,"dirs.run_cpu_time_us":860,"dirs.run_wall_time_us":5115,"drs_written":1,"lbm_read_time_us":1640,"lbm_reads_lt_1ms":67,"lbm_write_time_us":934,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":21}
I20260807 07:53:08.353664 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1110 bytes on disk
I20260807 07:53:08.354421 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.355003 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:08.360775 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.030s sys 0.008s Metrics: {"bytes_written":10027,"cfile_cache_hit":13,"cfile_cache_hit_bytes":4890,"cfile_cache_miss":31,"cfile_cache_miss_bytes":8004,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":4634,"dirs.run_cpu_time_us":960,"dirs.run_wall_time_us":6419,"drs_written":1,"lbm_read_time_us":1209,"lbm_reads_lt_1ms":63,"lbm_write_time_us":853,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":21,"thread_start_us":20285,"threads_started":1}
I20260807 07:53:08.361665 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1110 bytes on disk
I20260807 07:53:08.362191 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.362795 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:08.366953 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":156,"dirs.run_wall_time_us":1053,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":713,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:08.367635 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 384 bytes on disk
I20260807 07:53:08.368645 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.369483 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.333016
I20260807 07:53:08.374533 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1310,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":841,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:08.375203 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 384 bytes on disk
I20260807 07:53:08.375777 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.376375 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.333016
I20260807 07:53:08.398034 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.024s sys 0.004s Metrics: {"bytes_written":10460,"cfile_cache_hit":12,"cfile_cache_hit_bytes":5236,"cfile_cache_miss":28,"cfile_cache_miss_bytes":8248,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":481,"dirs.run_cpu_time_us":1067,"dirs.run_wall_time_us":7736,"drs_written":1,"lbm_read_time_us":1159,"lbm_reads_lt_1ms":56,"lbm_write_time_us":720,"lbm_writes_lt_1ms":27,"mutex_wait_us":1158,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":23,"spinlock_wait_cycles":11520,"thread_start_us":5251,"threads_started":1}
I20260807 07:53:08.398979 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1203 bytes on disk
I20260807 07:53:08.399798 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":145,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.407819 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.021s sys 0.006s Metrics: {"bytes_written":10460,"cfile_cache_hit":12,"cfile_cache_hit_bytes":5236,"cfile_cache_miss":28,"cfile_cache_miss_bytes":8248,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2883,"dirs.run_cpu_time_us":1000,"dirs.run_wall_time_us":6647,"drs_written":1,"lbm_read_time_us":1031,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1015,"lbm_writes_lt_1ms":27,"mutex_wait_us":24,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":23,"spinlock_wait_cycles":12160,"thread_start_us":6511,"threads_started":1}
I20260807 07:53:08.408607 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1203 bytes on disk
I20260807 07:53:08.409219 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.411197 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36112:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.423854 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 22b02baff831466ca3ba50a76e2a8adc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.11 [id=1a3b286952e44b53ad4af2b8819e35ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.424331 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b02baff831466ca3ba50a76e2a8adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.429615 10215 tablet_bootstrap.cc:492] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.430958 10215 tablet_bootstrap.cc:654] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.433247 10215 tablet_bootstrap.cc:492] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.433457 10215 ts_tablet_manager.cc:1403] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.434111 10215 raft_consensus.cc:359] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.434319 10215 raft_consensus.cc:385] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.434412 10215 raft_consensus.cc:740] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.434679 10215 consensus_queue.cc:260] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.434846 10215 raft_consensus.cc:399] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.434973 10215 raft_consensus.cc:493] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.435087 10215 raft_consensus.cc:3060] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.437008 10215 raft_consensus.cc:515] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.437251 10215 leader_election.cc:304] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.437531 10215 leader_election.cc:290] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.437620 10219 raft_consensus.cc:2804] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.437805 10219 raft_consensus.cc:697] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.438061 10219 consensus_queue.cc:237] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.438336 10215 ts_tablet_manager.cc:1434] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:08.441032 9887 catalog_manager.cc:5697] T 22b02baff831466ca3ba50a76e2a8adc P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.476999 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36128:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.489113 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 19acc1cecf8b41088288bd8d748668bc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.12 [id=246c3f8399da405ba48f3c33dd19a445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.489660 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19acc1cecf8b41088288bd8d748668bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.494161 10215 tablet_bootstrap.cc:492] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.495502 10215 tablet_bootstrap.cc:654] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.497756 10215 tablet_bootstrap.cc:492] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.497896 10215 ts_tablet_manager.cc:1403] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.498521 10215 raft_consensus.cc:359] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.498669 10215 raft_consensus.cc:385] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.498812 10215 raft_consensus.cc:740] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.499076 10215 consensus_queue.cc:260] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.499251 10215 raft_consensus.cc:399] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.499352 10215 raft_consensus.cc:493] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.499498 10215 raft_consensus.cc:3060] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.501551 10215 raft_consensus.cc:515] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.501806 10215 leader_election.cc:304] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.502009 10215 leader_election.cc:290] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.502141 10219 raft_consensus.cc:2804] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.502350 10219 raft_consensus.cc:697] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.502526 10215 ts_tablet_manager.cc:1434] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260807 07:53:08.502600 10219 consensus_queue.cc:237] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.505504 9887 catalog_manager.cc:5697] T 19acc1cecf8b41088288bd8d748668bc P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.530795 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000076
I20260807 07:53:08.540603 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36142:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.548096 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.006s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1162,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1681,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:08.549127 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 548 bytes on disk
I20260807 07:53:08.550562 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.551323 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.407272
I20260807 07:53:08.554752 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 7c9640934d4a4d3d9ce563d2911c11dc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.13 [id=4b1f96fa0a2b4209b4e4826f8dcaf492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.555244 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c9640934d4a4d3d9ce563d2911c11dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.564181 10215 tablet_bootstrap.cc:492] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.566004 10215 tablet_bootstrap.cc:654] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.568032 10215 tablet_bootstrap.cc:492] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.568210 10215 ts_tablet_manager.cc:1403] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.569068 10215 raft_consensus.cc:359] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.569267 10215 raft_consensus.cc:385] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.569391 10215 raft_consensus.cc:740] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.569661 10215 consensus_queue.cc:260] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.569819 10215 raft_consensus.cc:399] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.569931 10215 raft_consensus.cc:493] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.570055 10215 raft_consensus.cc:3060] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.572176 10215 raft_consensus.cc:515] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.572407 10215 leader_election.cc:304] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.572634 10215 leader_election.cc:290] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.572743 10219 raft_consensus.cc:2804] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.572870 10219 raft_consensus.cc:697] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.573140 10215 ts_tablet_manager.cc:1434] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 07:53:08.573115 10219 consensus_queue.cc:237] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.577409 9887 catalog_manager.cc:5697] T 7c9640934d4a4d3d9ce563d2911c11dc P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.580242 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.021s sys 0.000s Metrics: {"bytes_written":11350,"cfile_cache_hit":12,"cfile_cache_hit_bytes":5971,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9311,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":347,"dirs.run_cpu_time_us":913,"dirs.run_wall_time_us":6204,"drs_written":1,"lbm_read_time_us":4266,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":903,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":27}
I20260807 07:53:08.581167 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1362 bytes on disk
I20260807 07:53:08.581837 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.582458 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260807 07:53:08.594705 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.002s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":110,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1094,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1003,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:08.595407 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.286736
I20260807 07:53:08.619210 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.017s sys 0.004s Metrics: {"bytes_written":11794,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6341,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9845,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":568,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1041,"drs_written":1,"lbm_read_time_us":1038,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1026,"lbm_writes_lt_1ms":27,"mutex_wait_us":3,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":29}
I20260807 07:53:08.620270 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1441 bytes on disk
I20260807 07:53:08.620939 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.623667 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36154:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.640516 10099 tablet_service.cc:1511] Processing CreateTablet for tablet c6dc7b9938df4809b56ba56a1e76f37e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.14 [id=b4508e7cf319426b9234deef6084d7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.641069 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dc7b9938df4809b56ba56a1e76f37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.646178 10215 tablet_bootstrap.cc:492] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.647706 10215 tablet_bootstrap.cc:654] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.649742 10215 tablet_bootstrap.cc:492] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.649889 10215 ts_tablet_manager.cc:1403] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 07:53:08.650565 10215 raft_consensus.cc:359] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.650763 10215 raft_consensus.cc:385] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.650875 10215 raft_consensus.cc:740] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.651059 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:08.651135 10215 consensus_queue.cc:260] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.651361 10215 raft_consensus.cc:399] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.651453 10215 raft_consensus.cc:493] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.651587 10215 raft_consensus.cc:3060] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.653928 10215 raft_consensus.cc:515] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.654166 10215 leader_election.cc:304] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.654378 10215 leader_election.cc:290] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.654475 10219 raft_consensus.cc:2804] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.654630 10219 raft_consensus.cc:697] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.654911 10219 consensus_queue.cc:237] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.654915 10215 ts_tablet_manager.cc:1434] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:08.658948 9887 catalog_manager.cc:5697] T c6dc7b9938df4809b56ba56a1e76f37e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.660317 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:08.664307 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":6463,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":992,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":782,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:08.665274 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 592 bytes on disk
I20260807 07:53:08.667044 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.667888 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.423011
I20260807 07:53:08.676026 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.006s sys 0.004s Metrics: {"bytes_written":6463,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":1006,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":931,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:08.676810 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.423011
I20260807 07:53:08.691138 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.017s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6662,"cfile_cache_miss":28,"cfile_cache_miss_bytes":10276,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":602,"dirs.run_cpu_time_us":1007,"dirs.run_wall_time_us":5164,"drs_written":1,"lbm_read_time_us":1057,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1168,"lbm_writes_lt_1ms":31,"mutex_wait_us":39,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":31,"spinlock_wait_cycles":10112,"thread_start_us":708,"threads_started":1,"update_count":1}
I20260807 07:53:08.692373 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1510 bytes on disk
I20260807 07:53:08.693033 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.700848 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.017s sys 0.004s Metrics: {"bytes_written":12227,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6662,"cfile_cache_miss":28,"cfile_cache_miss_bytes":10276,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":3111,"dirs.run_cpu_time_us":603,"dirs.run_wall_time_us":3571,"drs_written":1,"lbm_read_time_us":1202,"lbm_reads_lt_1ms":60,"lbm_write_time_us":893,"lbm_writes_lt_1ms":27,"mutex_wait_us":218,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":31}
I20260807 07:53:08.701900 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1524 bytes on disk
I20260807 07:53:08.702919 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.706861 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36164:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.719400 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 9437dc0c40bb44bca431f7965f0fe891 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.15 [id=8cac428877774cfbab34798758e7de50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.719939 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9437dc0c40bb44bca431f7965f0fe891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.724807 10215 tablet_bootstrap.cc:492] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.726444 10215 tablet_bootstrap.cc:654] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.728896 10215 tablet_bootstrap.cc:492] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.729082 10215 ts_tablet_manager.cc:1403] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:08.729799 10215 raft_consensus.cc:359] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.730022 10215 raft_consensus.cc:385] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.730123 10215 raft_consensus.cc:740] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.730384 10215 consensus_queue.cc:260] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.730567 10215 raft_consensus.cc:399] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.730661 10215 raft_consensus.cc:493] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.730728 10215 raft_consensus.cc:3060] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.732570 10215 raft_consensus.cc:515] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.732777 10215 leader_election.cc:304] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.733026 10215 leader_election.cc:290] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.733093 10219 raft_consensus.cc:2804] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.733248 10219 raft_consensus.cc:697] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.733543 10215 ts_tablet_manager.cc:1434] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 07:53:08.733546 10219 consensus_queue.cc:237] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.737849 9887 catalog_manager.cc:5697] T 9437dc0c40bb44bca431f7965f0fe891 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.773944 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36172:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.786101 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 9933f9fe6b0a4a82b59347123dbfb8ee (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.16 [id=2f9f9d3ce4de461398cd92749d036f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.786566 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9933f9fe6b0a4a82b59347123dbfb8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.791908 10215 tablet_bootstrap.cc:492] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.793500 10215 tablet_bootstrap.cc:654] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.795709 10215 tablet_bootstrap.cc:492] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.795883 10215 ts_tablet_manager.cc:1403] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 07:53:08.796542 10215 raft_consensus.cc:359] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.796706 10215 raft_consensus.cc:385] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.796820 10215 raft_consensus.cc:740] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.797056 10215 consensus_queue.cc:260] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.797242 10215 raft_consensus.cc:399] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.797372 10215 raft_consensus.cc:493] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.797488 10215 raft_consensus.cc:3060] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.799372 10215 raft_consensus.cc:515] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.799626 10215 leader_election.cc:304] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.799832 10215 leader_election.cc:290] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.799933 10219 raft_consensus.cc:2804] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.800158 10219 raft_consensus.cc:697] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.800367 10215 ts_tablet_manager.cc:1434] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 07:53:08.800393 10219 consensus_queue.cc:237] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.803781 9887 catalog_manager.cc:5697] T 9933f9fe6b0a4a82b59347123dbfb8ee P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.836410 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36188:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.848626 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 98f2a5813d9942e4b26477544f388fad (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.17 [id=5f4716467a114109a3f015ac0b203cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.849107 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f2a5813d9942e4b26477544f388fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.854043 10215 tablet_bootstrap.cc:492] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.855374 10215 tablet_bootstrap.cc:654] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.871934 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260807 07:53:08.873881 10215 tablet_bootstrap.cc:492] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.874083 10215 ts_tablet_manager.cc:1403] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.001s
I20260807 07:53:08.874981 10215 raft_consensus.cc:359] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.875180 10215 raft_consensus.cc:385] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.875265 10215 raft_consensus.cc:740] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.875526 10215 consensus_queue.cc:260] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.875707 10215 raft_consensus.cc:399] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.875819 10215 raft_consensus.cc:493] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.875928 10215 raft_consensus.cc:3060] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.878834 10215 raft_consensus.cc:515] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.879114 10215 leader_election.cc:304] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.879357 10215 leader_election.cc:290] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.879457 10219 raft_consensus.cc:2804] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.879626 10219 raft_consensus.cc:697] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.879895 10219 consensus_queue.cc:237] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.879875 10215 ts_tablet_manager.cc:1434] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:08.884313 9887 catalog_manager.cc:5697] T 98f2a5813d9942e4b26477544f388fad P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.887243 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.010s sys 0.002s Metrics: {"bytes_written":6461,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1943,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1038,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:08.888141 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 586 bytes on disk
I20260807 07:53:08.888792 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.890272 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.394634
I20260807 07:53:08.909284 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.007s sys 0.010s Metrics: {"bytes_written":13566,"cfile_cache_hit":12,"cfile_cache_hit_bytes":7777,"cfile_cache_miss":28,"cfile_cache_miss_bytes":11893,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":658,"dirs.run_cpu_time_us":189,"dirs.run_wall_time_us":1212,"drs_written":1,"lbm_read_time_us":684,"lbm_reads_lt_1ms":56,"lbm_write_time_us":700,"lbm_writes_lt_1ms":27,"mutex_wait_us":60,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":37}
I20260807 07:53:08.910105 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1774 bytes on disk
I20260807 07:53:08.910766 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.931612 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36194:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:08.944020 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260807 07:53:08.944876 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1d1578af774561802c1e1923de101d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.18 [id=931b138218c64faba62ca154c9e5499f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:08.945287 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1d1578af774561802c1e1923de101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:08.950383 10215 tablet_bootstrap.cc:492] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:08.951678 10215 tablet_bootstrap.cc:654] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:08.954067 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260807 07:53:08.954107 10215 tablet_bootstrap.cc:492] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:08.954252 10215 ts_tablet_manager.cc:1403] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 07:53:08.954941 10215 raft_consensus.cc:359] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.955114 10215 raft_consensus.cc:385] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:08.955233 10215 raft_consensus.cc:740] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:08.955474 10215 consensus_queue.cc:260] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.955665 10215 raft_consensus.cc:399] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:08.955763 10215 raft_consensus.cc:493] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:08.955600 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.006s sys 0.004s Metrics: {"bytes_written":6461,"cfile_init":1,"dirs.queue_time_us":111,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1235,"drs_written":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4,"lbm_write_time_us":667,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:08.955895 10215 raft_consensus.cc:3060] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:08.956589 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.361631
I20260807 07:53:08.958070 10215 raft_consensus.cc:515] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.958333 10215 leader_election.cc:304] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:08.958608 10215 leader_election.cc:290] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:08.958660 10219 raft_consensus.cc:2804] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:08.958868 10219 raft_consensus.cc:697] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:08.959102 10215 ts_tablet_manager.cc:1434] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:08.959128 10219 consensus_queue.cc:237] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:08.962348 9887 catalog_manager.cc:5697] T 0a1d1578af774561802c1e1923de101d P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:08.973601 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.010s sys 0.003s Metrics: {"bytes_written":6461,"cfile_init":1,"dirs.queue_time_us":94,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":2425,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1098,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:08.974547 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 588 bytes on disk
I20260807 07:53:08.975224 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:08.975980 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.366411
I20260807 07:53:08.994688 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.022s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":8139,"cfile_cache_miss":31,"cfile_cache_miss_bytes":12801,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":765,"dirs.run_cpu_time_us":231,"dirs.run_wall_time_us":987,"drs_written":1,"lbm_read_time_us":3674,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":66,"lbm_write_time_us":1204,"lbm_writes_lt_1ms":31,"mutex_wait_us":207,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":39,"update_count":1}
I20260807 07:53:08.995617 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1844 bytes on disk
I20260807 07:53:08.997815 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":154,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.002017 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.022s sys 0.000s Metrics: {"bytes_written":13998,"cfile_cache_hit":12,"cfile_cache_hit_bytes":8142,"cfile_cache_miss":28,"cfile_cache_miss_bytes":12428,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":622,"dirs.run_cpu_time_us":915,"dirs.run_wall_time_us":5264,"drs_written":1,"lbm_read_time_us":1048,"lbm_reads_lt_1ms":56,"lbm_write_time_us":862,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":39,"spinlock_wait_cycles":2432}
I20260807 07:53:09.003568 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1856 bytes on disk
I20260807 07:53:09.004372 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.015389 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36208:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.028429 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6faae00a1645f99f94f5c0fec6a2fe (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.19 [id=73442f81ac1b4b56a987f73ee4b17a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.028946 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6faae00a1645f99f94f5c0fec6a2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.033679 10215 tablet_bootstrap.cc:492] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.035036 10215 tablet_bootstrap.cc:654] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.037503 10215 tablet_bootstrap.cc:492] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.037678 10215 ts_tablet_manager.cc:1403] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260807 07:53:09.038352 10215 raft_consensus.cc:359] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.038546 10215 raft_consensus.cc:385] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.038651 10215 raft_consensus.cc:740] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.038905 10215 consensus_queue.cc:260] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.039070 10215 raft_consensus.cc:399] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.039201 10215 raft_consensus.cc:493] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.039301 10215 raft_consensus.cc:3060] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.041520 10215 raft_consensus.cc:515] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.041746 10215 leader_election.cc:304] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.041965 10215 leader_election.cc:290] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.042125 10219 raft_consensus.cc:2804] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.042325 10219 raft_consensus.cc:697] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.042562 10215 ts_tablet_manager.cc:1434] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260807 07:53:09.042660 10219 consensus_queue.cc:237] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.046141 9887 catalog_manager.cc:5697] T 6a6faae00a1645f99f94f5c0fec6a2fe P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.080790 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36224:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.097553 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 722777a77c7c4020b1ce19913c7c1215 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.20 [id=9970986052ff455a840a13f4b34ca7d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.098165 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722777a77c7c4020b1ce19913c7c1215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.103328 10215 tablet_bootstrap.cc:492] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.104883 10215 tablet_bootstrap.cc:654] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.107297 10215 tablet_bootstrap.cc:492] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.107491 10215 ts_tablet_manager.cc:1403] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:09.108209 10215 raft_consensus.cc:359] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.108410 10215 raft_consensus.cc:385] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.108497 10215 raft_consensus.cc:740] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.108762 10215 consensus_queue.cc:260] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.108963 10215 raft_consensus.cc:399] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.109050 10215 raft_consensus.cc:493] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.109179 10215 raft_consensus.cc:3060] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.112221 10215 raft_consensus.cc:515] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.112484 10215 leader_election.cc:304] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.112697 10215 leader_election.cc:290] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.112809 10219 raft_consensus.cc:2804] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.113072 10219 raft_consensus.cc:697] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.113415 10219 consensus_queue.cc:237] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.113584 10215 ts_tablet_manager.cc:1434] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 07:53:09.117306 9887 catalog_manager.cc:5697] T 722777a77c7c4020b1ce19913c7c1215 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.156405 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36234:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.161809 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000076
I20260807 07:53:09.175028 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a5c2d9ab5e84454b9e694ec61ee11d20 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.21 [id=1c9e3ce1b6814101bdf4b25e11787554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.175469 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5c2d9ab5e84454b9e694ec61ee11d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.176851 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.007s sys 0.004s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1155,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1015,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:09.177740 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 521 bytes on disk
I20260807 07:53:09.178269 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.178828 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.319512
I20260807 07:53:09.181555 10215 tablet_bootstrap.cc:492] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.183342 10215 tablet_bootstrap.cc:654] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.186043 10215 tablet_bootstrap.cc:492] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.186211 10215 ts_tablet_manager.cc:1403] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:09.186934 10215 raft_consensus.cc:359] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.187144 10215 raft_consensus.cc:385] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.187250 10215 raft_consensus.cc:740] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.187525 10215 consensus_queue.cc:260] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.187693 10215 raft_consensus.cc:399] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.187801 10215 raft_consensus.cc:493] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.187918 10215 raft_consensus.cc:3060] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.190773 10215 raft_consensus.cc:515] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.191056 10215 leader_election.cc:304] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.191305 10215 leader_election.cc:290] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.191425 10219 raft_consensus.cc:2804] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.191627 10219 raft_consensus.cc:697] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.191917 10219 consensus_queue.cc:237] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.192016 10215 ts_tablet_manager.cc:1434] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:09.197805 9886 catalog_manager.cc:5697] T a5c2d9ab5e84454b9e694ec61ee11d20 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.202355 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.014s sys 0.008s Metrics: {"bytes_written":14875,"cfile_cache_hit":12,"cfile_cache_hit_bytes":8931,"cfile_cache_miss":28,"cfile_cache_miss_bytes":13623,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1004,"dirs.run_cpu_time_us":882,"dirs.run_wall_time_us":5722,"drs_written":1,"lbm_read_time_us":1198,"lbm_reads_lt_1ms":56,"lbm_write_time_us":866,"lbm_writes_lt_1ms":27,"mutex_wait_us":182,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":43,"spinlock_wait_cycles":1792}
I20260807 07:53:09.203408 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2015 bytes on disk
I20260807 07:53:09.204310 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.205015 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260807 07:53:09.216037 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.001s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1053,"drs_written":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1494,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:09.216720 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:09.217211 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.217777 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.280275
I20260807 07:53:09.239467 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36236:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.240726 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.020s sys 0.000s Metrics: {"bytes_written":15316,"cfile_cache_hit":12,"cfile_cache_hit_bytes":9301,"cfile_cache_miss":28,"cfile_cache_miss_bytes":14159,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":653,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":1173,"drs_written":1,"lbm_read_time_us":839,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1028,"lbm_writes_lt_1ms":27,"mutex_wait_us":175,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":45,"spinlock_wait_cycles":3456}
I20260807 07:53:09.241624 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2094 bytes on disk
I20260807 07:53:09.242316 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.249174 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:09.255349 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260807 07:53:09.255501 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf314aa6b414c9198d0ff7b5a8e15bf (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.22 [id=cdd8cece45854f9d9daf7460523cb733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.255929 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf314aa6b414c9198d0ff7b5a8e15bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.263944 10215 tablet_bootstrap.cc:492] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.265724 10215 tablet_bootstrap.cc:654] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.268214 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.004s Metrics: {"bytes_written":6463,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1039,"drs_written":1,"lbm_read_time_us":151,"lbm_reads_lt_1ms":4,"lbm_write_time_us":883,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:09.269162 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.360514
I20260807 07:53:09.274590 10215 tablet_bootstrap.cc:492] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.274732 10215 ts_tablet_manager.cc:1403] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260807 07:53:09.274621 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.010s sys 0.003s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":939,"drs_written":1,"lbm_read_time_us":1266,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":1304,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260807 07:53:09.275375 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.365318
I20260807 07:53:09.275532 10215 raft_consensus.cc:359] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.275703 10215 raft_consensus.cc:385] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.275801 10215 raft_consensus.cc:740] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.276067 10215 consensus_queue.cc:260] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.276274 10215 raft_consensus.cc:399] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.276360 10215 raft_consensus.cc:493] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.276467 10215 raft_consensus.cc:3060] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.279532 10215 raft_consensus.cc:515] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.279770 10215 leader_election.cc:304] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.279987 10215 leader_election.cc:290] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.280077 10219 raft_consensus.cc:2804] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.280346 10219 raft_consensus.cc:697] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.280514 10215 ts_tablet_manager.cc:1434] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:09.280639 10219 consensus_queue.cc:237] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.284291 9886 catalog_manager.cc:5697] T 0bf314aa6b414c9198d0ff7b5a8e15bf P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.322943 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.020s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":14,"cfile_cache_hit_bytes":9539,"cfile_cache_miss":34,"cfile_cache_miss_bytes":15197,"cfile_init":10,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":836,"dirs.run_cpu_time_us":902,"dirs.run_wall_time_us":4332,"drs_written":1,"lbm_read_time_us":1480,"lbm_reads_lt_1ms":74,"lbm_write_time_us":985,"lbm_writes_lt_1ms":31,"mutex_wait_us":594,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":47,"update_count":1}
I20260807 07:53:09.323496 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.024s sys 0.006s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":9622,"cfile_cache_miss":28,"cfile_cache_miss_bytes":14614,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1627,"dirs.run_cpu_time_us":883,"dirs.run_wall_time_us":3884,"drs_written":1,"lbm_read_time_us":1187,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1168,"lbm_writes_lt_1ms":31,"mutex_wait_us":93,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":47,"spinlock_wait_cycles":28800,"thread_start_us":6164,"threads_started":1,"update_count":1}
I20260807 07:53:09.324196 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2162 bytes on disk
I20260807 07:53:09.324643 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2162 bytes on disk
I20260807 07:53:09.325178 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.325392 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.339578 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36250:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.352843 10099 tablet_service.cc:1511] Processing CreateTablet for tablet cde93e1c6f3048328b9f6faf9b9d8ca8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.23 [id=763a2ee99bd846cab0942ff46671e15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.353330 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cde93e1c6f3048328b9f6faf9b9d8ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.358160 10215 tablet_bootstrap.cc:492] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.359510 10215 tablet_bootstrap.cc:654] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.361874 10215 tablet_bootstrap.cc:492] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.362071 10215 ts_tablet_manager.cc:1403] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 07:53:09.362679 10215 raft_consensus.cc:359] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.362862 10215 raft_consensus.cc:385] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.362967 10215 raft_consensus.cc:740] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.363257 10215 consensus_queue.cc:260] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.363438 10215 raft_consensus.cc:399] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.363526 10215 raft_consensus.cc:493] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.363592 10215 raft_consensus.cc:3060] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.365734 10215 raft_consensus.cc:515] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.365918 10215 leader_election.cc:304] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.366125 10215 leader_election.cc:290] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.366257 10219 raft_consensus.cc:2804] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.366398 10219 raft_consensus.cc:697] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.366678 10215 ts_tablet_manager.cc:1434] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 07:53:09.366689 10219 consensus_queue.cc:237] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.370394 9886 catalog_manager.cc:5697] T cde93e1c6f3048328b9f6faf9b9d8ca8 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.403259 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36256:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.415266 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f7b04beaf6d44225a8bae0b216aa4316 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.24 [id=f4126655eb2e44fd91d63a64904b3df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.415755 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b04beaf6d44225a8bae0b216aa4316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.420428 10215 tablet_bootstrap.cc:492] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.422446 10215 tablet_bootstrap.cc:654] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.424940 10215 tablet_bootstrap.cc:492] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.425108 10215 ts_tablet_manager.cc:1403] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:09.425969 10215 raft_consensus.cc:359] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.426115 10215 raft_consensus.cc:385] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.426314 10215 raft_consensus.cc:740] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.426586 10215 consensus_queue.cc:260] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.426788 10215 raft_consensus.cc:399] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.426932 10215 raft_consensus.cc:493] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.427074 10215 raft_consensus.cc:3060] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.429224 10215 raft_consensus.cc:515] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.429529 10215 leader_election.cc:304] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.429797 10215 leader_election.cc:290] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.429895 10219 raft_consensus.cc:2804] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.430102 10219 raft_consensus.cc:697] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.430330 10219 consensus_queue.cc:237] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.430383 10215 ts_tablet_manager.cc:1434] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 07:53:09.434150 9886 catalog_manager.cc:5697] T f7b04beaf6d44225a8bae0b216aa4316 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.471671 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36258:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.484375 10099 tablet_service.cc:1511] Processing CreateTablet for tablet addf128063dd49ed992f7ebc2c0affac (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.25 [id=73affb9257574ac49fbb0a5ff0ef8644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.484817 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet addf128063dd49ed992f7ebc2c0affac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.489694 10215 tablet_bootstrap.cc:492] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.491309 10215 tablet_bootstrap.cc:654] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.493369 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260807 07:53:09.493996 10215 tablet_bootstrap.cc:492] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.494179 10215 ts_tablet_manager.cc:1403] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260807 07:53:09.494837 10215 raft_consensus.cc:359] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.495014 10215 raft_consensus.cc:385] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.495101 10215 raft_consensus.cc:740] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.495354 10215 consensus_queue.cc:260] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.495517 10215 raft_consensus.cc:399] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.495615 10215 raft_consensus.cc:493] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.495707 10215 raft_consensus.cc:3060] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.498183 10215 raft_consensus.cc:515] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.498420 10215 leader_election.cc:304] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.498652 10215 leader_election.cc:290] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.498739 10219 raft_consensus.cc:2804] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.498865 10219 raft_consensus.cc:697] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.499192 10215 ts_tablet_manager.cc:1434] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:09.499130 10219 consensus_queue.cc:237] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.504525 9887 catalog_manager.cc:5697] T addf128063dd49ed992f7ebc2c0affac P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.510572 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.011s sys 0.002s Metrics: {"bytes_written":6462,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1069,"drs_written":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1063,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:09.511502 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 586 bytes on disk
I20260807 07:53:09.512215 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.512960 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.345045
I20260807 07:53:09.537864 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.019s sys 0.005s Metrics: {"bytes_written":17078,"cfile_cache_hit":12,"cfile_cache_hit_bytes":10737,"cfile_cache_miss":28,"cfile_cache_miss_bytes":16207,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":527,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1073,"drs_written":1,"lbm_read_time_us":1239,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1014,"lbm_writes_lt_1ms":27,"mutex_wait_us":151,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":53}
I20260807 07:53:09.538828 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2409 bytes on disk
I20260807 07:53:09.540150 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.555236 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36260:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.568436 10099 tablet_service.cc:1511] Processing CreateTablet for tablet bfda5285d1274a93a67fc2b13861594a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.26 [id=61a41e10013b408aabe45585c876f5d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.568837 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfda5285d1274a93a67fc2b13861594a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.573709 10215 tablet_bootstrap.cc:492] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.575233 10215 tablet_bootstrap.cc:654] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.576187 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:09.576743 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260807 07:53:09.577566 10215 tablet_bootstrap.cc:492] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.577739 10215 ts_tablet_manager.cc:1403] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 07:53:09.578379 10215 raft_consensus.cc:359] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.578547 10215 raft_consensus.cc:385] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.578671 10215 raft_consensus.cc:740] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.578923 10215 consensus_queue.cc:260] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.579105 10215 raft_consensus.cc:399] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.579785 10215 raft_consensus.cc:493] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.579932 10215 raft_consensus.cc:3060] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.582073 10215 raft_consensus.cc:515] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.582332 10215 leader_election.cc:304] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.582538 10215 leader_election.cc:290] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.582613 10219 raft_consensus.cc:2804] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.582756 10219 raft_consensus.cc:697] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.582979 10219 consensus_queue.cc:237] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.584002 10215 ts_tablet_manager.cc:1434] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:53:09.590911 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.013s sys 0.000s Metrics: {"bytes_written":6462,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1089,"drs_written":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1207,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:09.592064 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.338621
I20260807 07:53:09.595198 9887 catalog_manager.cc:5697] T bfda5285d1274a93a67fc2b13861594a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.593425 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.004s sys 0.010s Metrics: {"bytes_written":6462,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":995,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1073,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:09.597889 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 585 bytes on disk
I20260807 07:53:09.599568 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.600363 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.338621
I20260807 07:53:09.625932 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.021s sys 0.009s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11099,"cfile_cache_miss":31,"cfile_cache_miss_bytes":17107,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1664,"dirs.run_cpu_time_us":822,"dirs.run_wall_time_us":4030,"drs_written":1,"lbm_read_time_us":1745,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1281,"lbm_writes_lt_1ms":31,"mutex_wait_us":191,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":55,"thread_start_us":1732,"threads_started":1,"update_count":1}
I20260807 07:53:09.627274 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2478 bytes on disk
I20260807 07:53:09.628122 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":258,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.628893 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:09.629562 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.021s sys 0.006s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11099,"cfile_cache_miss":31,"cfile_cache_miss_bytes":17107,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1365,"dirs.run_cpu_time_us":1083,"dirs.run_wall_time_us":4984,"drs_written":1,"lbm_read_time_us":1231,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1118,"lbm_writes_lt_1ms":31,"mutex_wait_us":175,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":55,"thread_start_us":2001,"threads_started":1,"update_count":2}
I20260807 07:53:09.630424 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2478 bytes on disk
I20260807 07:53:09.630926 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.638365 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.001s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":115,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:09.638837 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:09.646692 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":70,"lbm_read_time_us":147,"lbm_reads_lt_1ms":12,"lbm_write_time_us":116,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:09.663578 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36274:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.677371 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffd4d5e724d4e3da59e383280c205ef (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.27 [id=437b78b336944f96a36bcbe850849143]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.677897 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffd4d5e724d4e3da59e383280c205ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.682773 10215 tablet_bootstrap.cc:492] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.684247 10215 tablet_bootstrap.cc:654] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.686782 10215 tablet_bootstrap.cc:492] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.686978 10215 ts_tablet_manager.cc:1403] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:09.687650 10215 raft_consensus.cc:359] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.687848 10215 raft_consensus.cc:385] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.687951 10215 raft_consensus.cc:740] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.688220 10215 consensus_queue.cc:260] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.688381 10215 raft_consensus.cc:399] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.688498 10215 raft_consensus.cc:493] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.688621 10215 raft_consensus.cc:3060] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.691201 10215 raft_consensus.cc:515] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.691439 10215 leader_election.cc:304] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.691726 10215 leader_election.cc:290] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.691778 10219 raft_consensus.cc:2804] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.691982 10219 raft_consensus.cc:697] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.692281 10219 consensus_queue.cc:237] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.692276 10215 ts_tablet_manager.cc:1434] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 07:53:09.696646 9886 catalog_manager.cc:5697] T 5ffd4d5e724d4e3da59e383280c205ef P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.738638 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36290:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.751223 10099 tablet_service.cc:1511] Processing CreateTablet for tablet b6d646f4e8aa44eebd278d68b475c426 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.28 [id=db5144c59b8b44f1936dcb0841ff68b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.751708 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d646f4e8aa44eebd278d68b475c426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.756472 10215 tablet_bootstrap.cc:492] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.758185 10215 tablet_bootstrap.cc:654] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.760551 10215 tablet_bootstrap.cc:492] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.760726 10215 ts_tablet_manager.cc:1403] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:09.761674 10215 raft_consensus.cc:359] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.761884 10215 raft_consensus.cc:385] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.762012 10215 raft_consensus.cc:740] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.762275 10215 consensus_queue.cc:260] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.762481 10215 raft_consensus.cc:399] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.762585 10215 raft_consensus.cc:493] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.762722 10215 raft_consensus.cc:3060] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.764977 10215 raft_consensus.cc:515] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.765242 10215 leader_election.cc:304] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.765518 10215 leader_election.cc:290] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.765650 10219 raft_consensus.cc:2804] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.765846 10219 raft_consensus.cc:697] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.766103 10219 consensus_queue.cc:237] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.766410 10215 ts_tablet_manager.cc:1434] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:09.770318 9886 catalog_manager.cc:5697] T b6d646f4e8aa44eebd278d68b475c426 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.791249 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260807 07:53:09.807976 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36304:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.808441 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.009s sys 0.005s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1350,"drs_written":1,"lbm_read_time_us":209,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1880,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:09.809226 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.312729
I20260807 07:53:09.823324 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f3dcfd4f04824cfc9e50b1bd3505397b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.29 [id=c4865a6c565743958430220d63bbdd8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.823874 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3dcfd4f04824cfc9e50b1bd3505397b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.831722 10215 tablet_bootstrap.cc:492] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.833302 10215 tablet_bootstrap.cc:654] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.837047 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.020s sys 0.006s Metrics: {"bytes_written":18388,"cfile_cache_hit":12,"cfile_cache_hit_bytes":11891,"cfile_cache_miss":28,"cfile_cache_miss_bytes":17939,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":968,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1194,"drs_written":1,"lbm_read_time_us":1438,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1096,"lbm_writes_lt_1ms":27,"mutex_wait_us":216,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":59,"spinlock_wait_cycles":768}
I20260807 07:53:09.838276 10215 tablet_bootstrap.cc:492] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.838308 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260807 07:53:09.838474 10215 ts_tablet_manager.cc:1403] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260807 07:53:09.839241 10215 raft_consensus.cc:359] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.839476 10215 raft_consensus.cc:385] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.839576 10215 raft_consensus.cc:740] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.839830 10215 consensus_queue.cc:260] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.839967 10215 raft_consensus.cc:399] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.840090 10215 raft_consensus.cc:493] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.840188 10215 raft_consensus.cc:3060] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.842159 10215 raft_consensus.cc:515] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.842363 10215 leader_election.cc:304] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.842589 10215 leader_election.cc:290] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.842692 10219 raft_consensus.cc:2804] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.842891 10219 raft_consensus.cc:697] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.843101 10215 ts_tablet_manager.cc:1434] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 07:53:09.843215 10219 consensus_queue.cc:237] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.848804 9886 catalog_manager.cc:5697] T f3dcfd4f04824cfc9e50b1bd3505397b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.850463 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.002s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":2939,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":920,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:09.851367 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.229212
I20260807 07:53:09.876312 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.016s sys 0.008s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12223,"cfile_cache_miss":28,"cfile_cache_miss_bytes":18179,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":823,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1051,"drs_written":1,"lbm_read_time_us":1405,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1156,"lbm_writes_lt_1ms":31,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":61,"spinlock_wait_cycles":4608,"update_count":1}
I20260807 07:53:09.877357 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2745 bytes on disk
I20260807 07:53:09.878142 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":124,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.881742 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:09.892774 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36320:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:09.894712 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.001s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":207,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":1099,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":947,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:09.895529 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260807 07:53:09.896350 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.897617 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.308281
I20260807 07:53:09.897781 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:09.910714 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":176,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":2673,"drs_written":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4,"lbm_write_time_us":970,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:09.911738 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 520 bytes on disk
I20260807 07:53:09.912561 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.913419 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.308281
I20260807 07:53:09.914314 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 9107501187214aab8c9ba5596baa0cb2 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.30 [id=024098a69d604a2fb6e7c804b1caea18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:09.914743 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9107501187214aab8c9ba5596baa0cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:09.920083 10215 tablet_bootstrap.cc:492] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:09.922326 10215 tablet_bootstrap.cc:654] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:09.925937 10215 tablet_bootstrap.cc:492] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:09.926107 10215 ts_tablet_manager.cc:1403] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:09.926847 10215 raft_consensus.cc:359] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.927014 10215 raft_consensus.cc:385] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:09.927140 10215 raft_consensus.cc:740] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:09.927413 10215 consensus_queue.cc:260] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.927588 10215 raft_consensus.cc:399] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:09.927668 10215 raft_consensus.cc:493] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:09.927778 10215 raft_consensus.cc:3060] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:09.929750 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.023s sys 0.004s Metrics: {"bytes_written":18826,"cfile_cache_hit":13,"cfile_cache_hit_bytes":12253,"cfile_cache_miss":31,"cfile_cache_miss_bytes":18887,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":5201,"dirs.run_cpu_time_us":952,"dirs.run_wall_time_us":8204,"drs_written":1,"lbm_read_time_us":1435,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1211,"lbm_writes_lt_1ms":27,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":61,"spinlock_wait_cycles":5632,"thread_start_us":1089,"threads_started":1}
I20260807 07:53:09.930850 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260807 07:53:09.931440 10215 raft_consensus.cc:515] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.931695 10215 leader_election.cc:304] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:09.931936 10215 leader_election.cc:290] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:09.931998 10219 raft_consensus.cc:2804] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:09.932178 10219 raft_consensus.cc:697] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:09.932432 10219 consensus_queue.cc:237] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:09.937616 9886 catalog_manager.cc:5697] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:09.943482 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.020s sys 0.008s Metrics: {"bytes_written":18826,"cfile_cache_hit":13,"cfile_cache_hit_bytes":12253,"cfile_cache_miss":31,"cfile_cache_miss_bytes":18887,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":382,"dirs.run_cpu_time_us":928,"dirs.run_wall_time_us":4302,"drs_written":1,"lbm_read_time_us":1572,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1268,"lbm_writes_lt_1ms":27,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":61,"spinlock_wait_cycles":9600}
I20260807 07:53:09.944453 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2756 bytes on disk
I20260807 07:53:09.945184 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.945916 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260807 07:53:09.947564 10215 ts_tablet_manager.cc:1434] T 9107501187214aab8c9ba5596baa0cb2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.021s user 0.001s sys 0.005s
I20260807 07:53:09.948908 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.006s sys 0.006s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":8702,"drs_written":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1076,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:09.950138 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.230365
I20260807 07:53:09.961858 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.012s sys 0.003s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":1411,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1175,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:09.962821 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:09.973659 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":114,"lbm_read_time_us":452,"lbm_reads_lt_1ms":21,"lbm_write_time_us":324,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:09.974342 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:09.974993 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:09.975790 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.247510
I20260807 07:53:09.983192 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.021s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12598,"cfile_cache_miss":28,"cfile_cache_miss_bytes":18720,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":834,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1270,"drs_written":1,"lbm_read_time_us":3519,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":63,"lbm_write_time_us":2617,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":63,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":63,"spinlock_wait_cycles":3968,"update_count":1}
I20260807 07:53:09.984239 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2839 bytes on disk
I20260807 07:53:09.984826 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.008006 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36334:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.010370 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.024s sys 0.004s Metrics: {"bytes_written":19259,"cfile_cache_hit":13,"cfile_cache_hit_bytes":12666,"cfile_cache_miss":31,"cfile_cache_miss_bytes":19178,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2073,"dirs.run_cpu_time_us":938,"dirs.run_wall_time_us":4714,"drs_written":1,"lbm_read_time_us":1505,"lbm_reads_lt_1ms":63,"lbm_write_time_us":986,"lbm_writes_lt_1ms":27,"mutex_wait_us":59,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":63,"spinlock_wait_cycles":1920,"thread_start_us":755,"threads_started":2}
I20260807 07:53:10.011332 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2858 bytes on disk
I20260807 07:53:10.012094 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.021565 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 27966960ed174baa8497b77dc434d4f5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.31 [id=281f44c370f54764ad0a92e8571c0d92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.022054 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27966960ed174baa8497b77dc434d4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.026876 10215 tablet_bootstrap.cc:492] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.028574 10215 tablet_bootstrap.cc:654] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.031186 10215 tablet_bootstrap.cc:492] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.031409 10215 ts_tablet_manager.cc:1403] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 07:53:10.032114 10215 raft_consensus.cc:359] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.032307 10215 raft_consensus.cc:385] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.032404 10215 raft_consensus.cc:740] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.032728 10215 consensus_queue.cc:260] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.032905 10215 raft_consensus.cc:399] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.033018 10215 raft_consensus.cc:493] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.033142 10215 raft_consensus.cc:3060] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.035717 10215 raft_consensus.cc:515] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.035949 10215 leader_election.cc:304] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.036214 10215 leader_election.cc:290] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.036319 10220 raft_consensus.cc:2804] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.036592 10220 raft_consensus.cc:697] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.036806 10215 ts_tablet_manager.cc:1434] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260807 07:53:10.036917 10220 consensus_queue.cc:237] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.041584 9886 catalog_manager.cc:5697] T 27966960ed174baa8497b77dc434d4f5 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.082666 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36340:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.094884 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc57f7143874e1fb00252487bf21090 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.32 [id=6cc58ed77ef24b5290cf3179edcaff0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.095363 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc57f7143874e1fb00252487bf21090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.099679 10215 tablet_bootstrap.cc:492] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.101096 10215 tablet_bootstrap.cc:654] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.103636 10215 tablet_bootstrap.cc:492] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.103794 10215 ts_tablet_manager.cc:1403] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:10.104421 10215 raft_consensus.cc:359] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.104586 10215 raft_consensus.cc:385] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.104702 10215 raft_consensus.cc:740] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.104955 10215 consensus_queue.cc:260] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.105145 10215 raft_consensus.cc:399] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.105228 10215 raft_consensus.cc:493] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.105378 10215 raft_consensus.cc:3060] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.107731 10215 raft_consensus.cc:515] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.107967 10215 leader_election.cc:304] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.108201 10215 leader_election.cc:290] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.108263 10220 raft_consensus.cc:2804] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.108465 10220 raft_consensus.cc:697] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.108690 10220 consensus_queue.cc:237] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.112596 9887 catalog_manager.cc:5697] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.116531 10215 ts_tablet_manager.cc:1434] T 4cc57f7143874e1fb00252487bf21090 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.001s sys 0.004s
I20260807 07:53:10.129531 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260807 07:53:10.145320 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.010s sys 0.001s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1910,"drs_written":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1083,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:10.146236 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 562 bytes on disk
I20260807 07:53:10.146844 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.147439 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.278352
I20260807 07:53:10.185590 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.019s sys 0.015s Metrics: {"bytes_written":20139,"cfile_cache_hit":13,"cfile_cache_hit_bytes":13368,"cfile_cache_miss":31,"cfile_cache_miss_bytes":20522,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1284,"dirs.run_cpu_time_us":838,"dirs.run_wall_time_us":4625,"drs_written":1,"lbm_read_time_us":1664,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1409,"lbm_writes_lt_1ms":27,"mutex_wait_us":842,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":67,"spinlock_wait_cycles":2432}
I20260807 07:53:10.186798 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3036 bytes on disk
I20260807 07:53:10.187556 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.187459 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36354:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.202239 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 59943b275bc742428589c5e0bda94a57 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.33 [id=df6e072310b842c1beb67efc39e9f97e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.202883 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59943b275bc742428589c5e0bda94a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.208863 10215 tablet_bootstrap.cc:492] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.211336 10215 tablet_bootstrap.cc:654] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.214224 10215 tablet_bootstrap.cc:492] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.214485 10215 ts_tablet_manager.cc:1403] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:10.215539 10215 raft_consensus.cc:359] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.215755 10215 raft_consensus.cc:385] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.215888 10215 raft_consensus.cc:740] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.216204 10215 consensus_queue.cc:260] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.216430 10215 raft_consensus.cc:399] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.216657 10215 raft_consensus.cc:493] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.216820 10215 raft_consensus.cc:3060] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.219300 10215 raft_consensus.cc:515] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.219650 10215 leader_election.cc:304] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.220005 10215 leader_election.cc:290] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.220150 10220 raft_consensus.cc:2804] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.220387 10220 raft_consensus.cc:697] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.220656 10215 ts_tablet_manager.cc:1434] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:10.220799 10220 consensus_queue.cc:237] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.225808 9887 catalog_manager.cc:5697] T 59943b275bc742428589c5e0bda94a57 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.235975 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:10.252102 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.009s sys 0.004s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":77,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1271,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1382,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:10.253237 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.284371
I20260807 07:53:10.263159 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:10.294456 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.012s sys 0.008s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":1129,"drs_written":1,"lbm_read_time_us":125,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1715,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:10.306248 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 550 bytes on disk
I20260807 07:53:10.307057 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.307806 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.274609
I20260807 07:53:10.324543 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36370:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.328836 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.075s user 0.022s sys 0.022s Metrics: {"bytes_written":20585,"cfile_cache_hit":14,"cfile_cache_hit_bytes":13764,"cfile_cache_miss":34,"cfile_cache_miss_bytes":21450,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1396,"dirs.run_cpu_time_us":1124,"dirs.run_wall_time_us":6858,"drs_written":1,"lbm_read_time_us":1501,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1399,"lbm_writes_lt_1ms":27,"mutex_wait_us":51,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":69,"spinlock_wait_cycles":2176,"thread_start_us":11078,"threads_started":1}
I20260807 07:53:10.331007 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3117 bytes on disk
I20260807 07:53:10.332562 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.344209 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 075cc1d3a8ad402688154efcfc2f026b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.34 [id=ad3114e3e77e4d079176a3860c227e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.344781 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 075cc1d3a8ad402688154efcfc2f026b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.351182 10215 tablet_bootstrap.cc:492] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.353870 10215 tablet_bootstrap.cc:654] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.357120 10215 tablet_bootstrap.cc:492] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.357307 10215 ts_tablet_manager.cc:1403] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:53:10.358009 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.018s sys 0.011s Metrics: {"bytes_written":20585,"cfile_cache_hit":12,"cfile_cache_hit_bytes":13736,"cfile_cache_miss":28,"cfile_cache_miss_bytes":20686,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1036,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":2344,"drs_written":1,"lbm_read_time_us":1248,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1474,"lbm_writes_lt_1ms":27,"mutex_wait_us":67,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":69,"spinlock_wait_cycles":2816}
I20260807 07:53:10.359069 10215 raft_consensus.cc:359] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.359104 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000018
I20260807 07:53:10.359344 10215 raft_consensus.cc:385] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.359578 10215 raft_consensus.cc:740] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.359884 10215 consensus_queue.cc:260] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.360105 10215 raft_consensus.cc:399] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.360201 10215 raft_consensus.cc:493] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.360350 10215 raft_consensus.cc:3060] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.363535 10215 raft_consensus.cc:515] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.363907 10215 leader_election.cc:304] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.364287 10215 leader_election.cc:290] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.364380 10220 raft_consensus.cc:2804] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.364634 10220 raft_consensus.cc:697] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.364974 10220 consensus_queue.cc:237] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.365083 10215 ts_tablet_manager.cc:1434] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260807 07:53:10.371721 9887 catalog_manager.cc:5697] T 075cc1d3a8ad402688154efcfc2f026b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.378261 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.011s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":999,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1103,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:10.379527 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191462
I20260807 07:53:10.408617 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.023s sys 0.005s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":14078,"cfile_cache_miss":28,"cfile_cache_miss_bytes":20936,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":464,"dirs.run_cpu_time_us":1037,"dirs.run_wall_time_us":4838,"drs_written":1,"lbm_read_time_us":1306,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1070,"lbm_writes_lt_1ms":31,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":71,"update_count":1}
I20260807 07:53:10.409725 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3192 bytes on disk
I20260807 07:53:10.410501 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.429488 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36382:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.438701 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:10.444361 10099 tablet_service.cc:1511] Processing CreateTablet for tablet ea237826e96e4553a9de2e875da7a9f4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.35 [id=98bf06d8020f426797e551f45a2440ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.444978 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea237826e96e4553a9de2e875da7a9f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.451110 10215 tablet_bootstrap.cc:492] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.452989 10215 tablet_bootstrap.cc:654] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.455780 10215 tablet_bootstrap.cc:492] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.455658 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.008s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1073,"drs_written":1,"lbm_read_time_us":47,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1332,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:10.455951 10215 ts_tablet_manager.cc:1403] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:10.456686 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 461 bytes on disk
I20260807 07:53:10.456736 10215 raft_consensus.cc:359] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.456926 10215 raft_consensus.cc:385] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.457027 10215 raft_consensus.cc:740] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.457317 10215 consensus_queue.cc:260] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.457528 10215 raft_consensus.cc:399] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.457645 10215 raft_consensus.cc:493] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.457764 10215 raft_consensus.cc:3060] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.457679 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":133,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.458406 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.296399
I20260807 07:53:10.461364 10215 raft_consensus.cc:515] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.461696 10215 leader_election.cc:304] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.462009 10215 leader_election.cc:290] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.462139 10220 raft_consensus.cc:2804] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.462390 10220 raft_consensus.cc:697] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.462759 10215 ts_tablet_manager.cc:1434] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260807 07:53:10.462792 10220 consensus_queue.cc:237] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.471100 9887 catalog_manager.cc:5697] T ea237826e96e4553a9de2e875da7a9f4 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.491627 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.021s sys 0.011s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":14359,"cfile_cache_miss":31,"cfile_cache_miss_bytes":21885,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1844,"dirs.run_cpu_time_us":1101,"dirs.run_wall_time_us":4983,"drs_written":1,"lbm_read_time_us":1284,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1418,"lbm_writes_lt_1ms":31,"mutex_wait_us":158,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":73,"spinlock_wait_cycles":14976,"thread_start_us":735,"threads_started":1,"update_count":1}
I20260807 07:53:10.492640 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3273 bytes on disk
I20260807 07:53:10.493284 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.516636 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36392:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.531657 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 30024869a6f648fa97bb9d151282eb7a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.36 [id=b655ac66ce24411c88b69305aaef7329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.532226 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30024869a6f648fa97bb9d151282eb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.537948 10215 tablet_bootstrap.cc:492] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.540534 10215 tablet_bootstrap.cc:654] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.543416 10215 tablet_bootstrap.cc:492] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.543628 10215 ts_tablet_manager.cc:1403] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:10.544531 10215 raft_consensus.cc:359] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.544744 10215 raft_consensus.cc:385] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.544961 10215 raft_consensus.cc:740] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.545258 10215 consensus_queue.cc:260] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.545539 10215 raft_consensus.cc:399] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.545682 10215 raft_consensus.cc:493] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.545943 10215 raft_consensus.cc:3060] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.548501 10215 raft_consensus.cc:515] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.548942 10215 leader_election.cc:304] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.549211 10215 leader_election.cc:290] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.549314 10220 raft_consensus.cc:2804] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.549602 10220 raft_consensus.cc:697] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.549959 10215 ts_tablet_manager.cc:1434] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 07:53:10.549958 10220 consensus_queue.cc:237] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.555697 9887 catalog_manager.cc:5697] T 30024869a6f648fa97bb9d151282eb7a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.585124 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000093
I20260807 07:53:10.599247 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.012s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":272,"dirs.run_wall_time_us":1274,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":921,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:10.600258 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.289588
I20260807 07:53:10.603344 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36400:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.618525 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 3458ee2bb6bc4f21b117b60602867058 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.37 [id=58afd0d5339a487d896c81f681789084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.619062 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3458ee2bb6bc4f21b117b60602867058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.624522 10215 tablet_bootstrap.cc:492] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.626121 10215 tablet_bootstrap.cc:654] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.629133 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.025s sys 0.003s Metrics: {"bytes_written":21894,"cfile_cache_hit":12,"cfile_cache_hit_bytes":14851,"cfile_cache_miss":28,"cfile_cache_miss_bytes":22271,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1190,"dirs.run_cpu_time_us":200,"dirs.run_wall_time_us":1447,"drs_written":1,"lbm_read_time_us":1471,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1182,"lbm_writes_lt_1ms":27,"mutex_wait_us":113,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":75}
I20260807 07:53:10.629812 10215 tablet_bootstrap.cc:492] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.629971 10215 ts_tablet_manager.cc:1403] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:10.630064 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260807 07:53:10.630753 10215 raft_consensus.cc:359] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.630937 10215 raft_consensus.cc:385] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.630998 10215 raft_consensus.cc:740] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.631309 10215 consensus_queue.cc:260] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.631479 10215 raft_consensus.cc:399] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.631572 10215 raft_consensus.cc:493] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.631654 10215 raft_consensus.cc:3060] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.634689 10215 raft_consensus.cc:515] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.634922 10215 leader_election.cc:304] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.635206 10220 raft_consensus.cc:2804] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.635332 10220 raft_consensus.cc:697] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.635639 10220 consensus_queue.cc:237] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.635923 10215 leader_election.cc:290] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.637480 10215 ts_tablet_manager.cc:1434] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 07:53:10.643419 9887 catalog_manager.cc:5697] T 3458ee2bb6bc4f21b117b60602867058 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.649096 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.010s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":157,"dirs.run_wall_time_us":1219,"drs_written":1,"lbm_read_time_us":605,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1187,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:10.650060 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.216478
I20260807 07:53:10.661463 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:10.679901 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.012s sys 0.001s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":452,"dirs.run_cpu_time_us":154,"dirs.run_wall_time_us":935,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1123,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:10.680233 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.029s sys 0.000s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":15183,"cfile_cache_miss":28,"cfile_cache_miss_bytes":22545,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1260,"dirs.run_cpu_time_us":441,"dirs.run_wall_time_us":2450,"drs_written":1,"lbm_read_time_us":1555,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1306,"lbm_writes_lt_1ms":31,"mutex_wait_us":240,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":77,"update_count":2}
I20260807 07:53:10.681085 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3439 bytes on disk
I20260807 07:53:10.681209 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 521 bytes on disk
I20260807 07:53:10.681780 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.682495 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.683063 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.300793
I20260807 07:53:10.705292 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36406:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.710425 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.025s sys 0.001s Metrics: {"bytes_written":22339,"cfile_cache_hit":14,"cfile_cache_hit_bytes":15244,"cfile_cache_miss":34,"cfile_cache_miss_bytes":23628,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":1583,"dirs.run_cpu_time_us":870,"dirs.run_wall_time_us":4458,"drs_written":1,"lbm_read_time_us":1406,"lbm_reads_lt_1ms":70,"lbm_write_time_us":685,"lbm_writes_lt_1ms":27,"mutex_wait_us":121,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":77,"thread_start_us":741,"threads_started":1}
I20260807 07:53:10.712107 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3464 bytes on disk
I20260807 07:53:10.712896 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.717909 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 3e844c14030245628098d75f1f2c488a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.38 [id=77b3a7a983f74881b0f88cd827cde650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.718365 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e844c14030245628098d75f1f2c488a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.723308 10215 tablet_bootstrap.cc:492] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.725113 10215 tablet_bootstrap.cc:654] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.727623 10215 tablet_bootstrap.cc:492] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.727767 10215 ts_tablet_manager.cc:1403] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:10.728348 10215 raft_consensus.cc:359] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.728561 10215 raft_consensus.cc:385] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.728660 10215 raft_consensus.cc:740] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.728920 10215 consensus_queue.cc:260] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.729068 10215 raft_consensus.cc:399] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.729130 10215 raft_consensus.cc:493] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.729224 10215 raft_consensus.cc:3060] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.731554 10215 raft_consensus.cc:515] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.731755 10215 leader_election.cc:304] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.731909 10215 leader_election.cc:290] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.731998 10220 raft_consensus.cc:2804] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.732250 10220 raft_consensus.cc:697] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.732422 10215 ts_tablet_manager.cc:1434] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 07:53:10.732512 10220 consensus_queue.cc:237] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.736749 9886 catalog_manager.cc:5697] T 3e844c14030245628098d75f1f2c488a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.744807 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:10.759166 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.000s Metrics: {"bytes_written":6067,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":1169,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":963,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:10.759923 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 554 bytes on disk
I20260807 07:53:10.760483 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.761080 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259755
I20260807 07:53:10.784780 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36408:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.790421 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.025s sys 0.000s Metrics: {"bytes_written":22766,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15588,"cfile_cache_miss":31,"cfile_cache_miss_bytes":23814,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1216,"drs_written":1,"lbm_read_time_us":1800,"lbm_reads_lt_1ms":63,"lbm_write_time_us":868,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":79}
I20260807 07:53:10.791839 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260807 07:53:10.799263 10099 tablet_service.cc:1511] Processing CreateTablet for tablet c508a6d05a61464d9a30050afa5989dc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.39 [id=cd393a40832c427fa409d48a0ce1ec66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.799737 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c508a6d05a61464d9a30050afa5989dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.805035 10215 tablet_bootstrap.cc:492] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.804801 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":147,"dirs.run_wall_time_us":1519,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1062,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"spinlock_wait_cycles":10496,"update_count":1}
I20260807 07:53:10.805770 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3712 bytes on disk
I20260807 07:53:10.806392 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":106,"lbm_reads_lt_1ms":8}
I20260807 07:53:10.806823 10215 tablet_bootstrap.cc:654] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.807153 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:10.809620 10215 tablet_bootstrap.cc:492] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.809818 10215 ts_tablet_manager.cc:1403] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:10.810458 10215 raft_consensus.cc:359] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.810616 10215 raft_consensus.cc:385] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.810709 10215 raft_consensus.cc:740] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.810969 10215 consensus_queue.cc:260] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.811132 10215 raft_consensus.cc:399] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.811250 10215 raft_consensus.cc:493] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.811367 10215 raft_consensus.cc:3060] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.813472 10215 raft_consensus.cc:515] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.813737 10215 leader_election.cc:304] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.814005 10215 leader_election.cc:290] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.814096 10220 raft_consensus.cc:2804] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.814263 10220 raft_consensus.cc:697] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.814543 10220 consensus_queue.cc:237] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.814904 10215 ts_tablet_manager.cc:1434] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260807 07:53:10.817098 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":180,"lbm_read_time_us":475,"lbm_reads_lt_1ms":21,"lbm_write_time_us":294,"lbm_writes_lt_1ms":9,"mutex_wait_us":31,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":7552,"update_count":1}
I20260807 07:53:10.817737 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.221472
I20260807 07:53:10.821169 9886 catalog_manager.cc:5697] T c508a6d05a61464d9a30050afa5989dc P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.845641 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.025s sys 0.001s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15956,"cfile_cache_miss":31,"cfile_cache_miss_bytes":23720,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1594,"dirs.run_cpu_time_us":1068,"dirs.run_wall_time_us":4622,"drs_written":1,"lbm_read_time_us":1305,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1052,"lbm_writes_lt_1ms":31,"mutex_wait_us":254,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":81,"spinlock_wait_cycles":4736,"thread_start_us":624,"threads_started":1,"update_count":1}
I20260807 07:53:10.846758 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3613 bytes on disk
I20260807 07:53:10.847373 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260807 07:53:10.847900 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260807 07:53:10.852092 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.001s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":113,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:10.852504 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:10.858353 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":56,"lbm_read_time_us":114,"lbm_reads_lt_1ms":8,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:10.874085 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36424:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.887497 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 6162579cb5bb463d9d6ef31bdf7b5f9d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.40 [id=1a144fcf24ca44fc8dee7fcf269be54a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.887954 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6162579cb5bb463d9d6ef31bdf7b5f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.892606 10215 tablet_bootstrap.cc:492] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.893980 10215 tablet_bootstrap.cc:654] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.896212 10215 tablet_bootstrap.cc:492] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.896376 10215 ts_tablet_manager.cc:1403] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 07:53:10.897035 10215 raft_consensus.cc:359] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.897204 10215 raft_consensus.cc:385] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.897321 10215 raft_consensus.cc:740] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.897626 10215 consensus_queue.cc:260] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.897827 10215 raft_consensus.cc:399] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.897920 10215 raft_consensus.cc:493] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.898051 10215 raft_consensus.cc:3060] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.899945 10215 raft_consensus.cc:515] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.900195 10215 leader_election.cc:304] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.900403 10215 leader_election.cc:290] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.900480 10220 raft_consensus.cc:2804] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.900708 10220 raft_consensus.cc:697] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.900971 10220 consensus_queue.cc:237] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.901255 10215 ts_tablet_manager.cc:1434] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 07:53:10.907100 9886 catalog_manager.cc:5697] T 6162579cb5bb463d9d6ef31bdf7b5f9d P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.932780 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:10.947386 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.001s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":594,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1220,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1056,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:10.948462 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.285330
I20260807 07:53:10.953809 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36430:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:10.963973 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:10.971622 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a2f235b507d448fd81c78a3cce6dccb5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.41 [id=be8608d9c5844718bf128adceb692d92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:10.972153 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f235b507d448fd81c78a3cce6dccb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:10.975692 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.001s Metrics: {"bytes_written":6349,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1013,"drs_written":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4,"lbm_write_time_us":782,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260807 07:53:10.976591 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.294412
I20260807 07:53:10.979367 10215 tablet_bootstrap.cc:492] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:10.981172 10215 tablet_bootstrap.cc:654] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:10.982823 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.018s sys 0.008s Metrics: {"bytes_written":23642,"cfile_cache_hit":13,"cfile_cache_hit_bytes":16328,"cfile_cache_miss":31,"cfile_cache_miss_bytes":24986,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1154,"dirs.run_cpu_time_us":619,"dirs.run_wall_time_us":3000,"drs_written":1,"lbm_read_time_us":1465,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1069,"lbm_writes_lt_1ms":27,"mutex_wait_us":506,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":83}
I20260807 07:53:10.983906 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260807 07:53:10.986892 10215 tablet_bootstrap.cc:492] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:10.987066 10215 ts_tablet_manager.cc:1403] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 07:53:10.987982 10215 raft_consensus.cc:359] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.988385 10215 raft_consensus.cc:385] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:10.988484 10215 raft_consensus.cc:740] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:10.989326 10215 consensus_queue.cc:260] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.989542 10215 raft_consensus.cc:399] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:10.989652 10215 raft_consensus.cc:493] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:10.989748 10215 raft_consensus.cc:3060] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:10.992601 10215 raft_consensus.cc:515] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.992857 10215 leader_election.cc:304] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:10.993103 10215 leader_election.cc:290] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:10.993188 10220 raft_consensus.cc:2804] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:10.993472 10220 raft_consensus.cc:697] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:10.993788 10220 consensus_queue.cc:237] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:10.994513 10215 ts_tablet_manager.cc:1434] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 07:53:10.994554 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.003s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":158,"dirs.run_wall_time_us":941,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":673,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:10.995558 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3963 bytes on disk
I20260807 07:53:10.998848 9886 catalog_manager.cc:5697] T a2f235b507d448fd81c78a3cce6dccb5 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:10.999089 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":186,"lbm_reads_lt_1ms":8}
I20260807 07:53:10.999974 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.213369
I20260807 07:53:11.019749 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.016s sys 0.011s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":16543,"cfile_cache_miss":28,"cfile_cache_miss_bytes":24775,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":550,"dirs.run_cpu_time_us":777,"dirs.run_wall_time_us":4425,"drs_written":1,"lbm_read_time_us":2048,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1195,"lbm_writes_lt_1ms":31,"mutex_wait_us":32,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":85,"spinlock_wait_cycles":14592,"update_count":1}
I20260807 07:53:11.020876 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3752 bytes on disk
I20260807 07:53:11.022306 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.022904 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:11.028115 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.000s Metrics: {"bytes_written":390,"delete_count":0,"lbm_write_time_us":149,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:11.028599 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:11.031056 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.023s sys 0.004s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":16663,"cfile_cache_miss":28,"cfile_cache_miss_bytes":24755,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":3510,"dirs.run_cpu_time_us":811,"dirs.run_wall_time_us":5381,"drs_written":1,"lbm_read_time_us":962,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1173,"lbm_writes_lt_1ms":31,"mutex_wait_us":518,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":85,"spinlock_wait_cycles":1280,"update_count":1}
I20260807 07:53:11.032184 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:11.035918 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.001s sys 0.003s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":518,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":69,"lbm_read_time_us":198,"lbm_reads_lt_1ms":8,"lbm_write_time_us":169,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260807 07:53:11.037529 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.004s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:11.038009 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:11.043438 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":64,"lbm_read_time_us":123,"lbm_reads_lt_1ms":12,"lbm_write_time_us":121,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:11.043923 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3796 bytes on disk
I20260807 07:53:11.044390 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.059792 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36446:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.072992 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0d652d6b7db94ae1ab998ded61eea080 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.42 [id=66a302ddceb743949d61d7833444a2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.073578 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d652d6b7db94ae1ab998ded61eea080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.078759 10215 tablet_bootstrap.cc:492] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.080173 10215 tablet_bootstrap.cc:654] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.082775 10215 tablet_bootstrap.cc:492] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.082929 10215 ts_tablet_manager.cc:1403] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 07:53:11.083577 10215 raft_consensus.cc:359] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.083714 10215 raft_consensus.cc:385] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.083844 10215 raft_consensus.cc:740] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.084081 10215 consensus_queue.cc:260] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.084260 10215 raft_consensus.cc:399] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.084362 10215 raft_consensus.cc:493] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.084504 10215 raft_consensus.cc:3060] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.087945 10215 raft_consensus.cc:515] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.088233 10215 leader_election.cc:304] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.088474 10215 leader_election.cc:290] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.088570 10220 raft_consensus.cc:2804] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.089308 10220 raft_consensus.cc:697] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.089617 10220 consensus_queue.cc:237] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.089732 10215 ts_tablet_manager.cc:1434] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 07:53:11.094494 9886 catalog_manager.cc:5697] T 0d652d6b7db94ae1ab998ded61eea080 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.109163 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:11.120440 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1332,"drs_written":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4,"lbm_write_time_us":821,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:11.121210 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 551 bytes on disk
I20260807 07:53:11.121840 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.122447 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.280922
I20260807 07:53:11.142305 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36456:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.157413 10099 tablet_service.cc:1511] Processing CreateTablet for tablet b25bd1839bc24afa9fb766bf8c2d6545 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.43 [id=ecc38019571741a69e02e519ed02a7e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.157959 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b25bd1839bc24afa9fb766bf8c2d6545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.165158 10215 tablet_bootstrap.cc:492] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.166998 10215 tablet_bootstrap.cc:654] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.169478 10215 tablet_bootstrap.cc:492] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.169677 10215 ts_tablet_manager.cc:1403] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:11.170316 10215 raft_consensus.cc:359] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.170526 10215 raft_consensus.cc:385] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.170609 10215 raft_consensus.cc:740] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.170905 10215 consensus_queue.cc:260] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.171111 10215 raft_consensus.cc:399] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.171237 10215 raft_consensus.cc:493] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.171355 10215 raft_consensus.cc:3060] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.174090 10215 raft_consensus.cc:515] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.174319 10215 leader_election.cc:304] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.174571 10215 leader_election.cc:290] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.174651 10220 raft_consensus.cc:2804] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.174800 10220 raft_consensus.cc:697] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.175069 10220 consensus_queue.cc:237] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.175264 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.033s sys 0.004s Metrics: {"bytes_written":24515,"cfile_cache_hit":13,"cfile_cache_hit_bytes":17068,"cfile_cache_miss":31,"cfile_cache_miss_bytes":26068,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1495,"dirs.run_cpu_time_us":845,"dirs.run_wall_time_us":3933,"drs_written":1,"lbm_read_time_us":2191,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1324,"lbm_writes_lt_1ms":27,"mutex_wait_us":51,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":87,"spinlock_wait_cycles":6528}
I20260807 07:53:11.176467 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000016
I20260807 07:53:11.176478 10215 ts_tablet_manager.cc:1434] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260807 07:53:11.183058 9886 catalog_manager.cc:5697] T b25bd1839bc24afa9fb766bf8c2d6545 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.194796 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.013s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":1703,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":1136,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1021,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:11.195669 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4143 bytes on disk
I20260807 07:53:11.196533 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":155,"lbm_reads_lt_1ms":8}
I20260807 07:53:11.197438 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.210593
I20260807 07:53:11.225445 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.027s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":17403,"cfile_cache_miss":28,"cfile_cache_miss_bytes":25859,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1084,"dirs.run_cpu_time_us":846,"dirs.run_wall_time_us":3516,"drs_written":1,"lbm_read_time_us":1803,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1075,"lbm_writes_lt_1ms":31,"mutex_wait_us":539,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":89,"update_count":1}
I20260807 07:53:11.226375 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3988 bytes on disk
I20260807 07:53:11.226999 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.253067 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36472:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.269443 10099 tablet_service.cc:1511] Processing CreateTablet for tablet bc47f50c6ffe4a2c85c5b821e0910e8f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.44 [id=fa4396f044b2458da078179e0b6b0881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.269979 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc47f50c6ffe4a2c85c5b821e0910e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.279633 10215 tablet_bootstrap.cc:492] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.282408 10215 tablet_bootstrap.cc:654] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.287046 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260807 07:53:11.290272 10215 tablet_bootstrap.cc:492] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.290642 10215 ts_tablet_manager.cc:1403] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260807 07:53:11.292503 10215 raft_consensus.cc:359] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.292743 10215 raft_consensus.cc:385] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.292951 10215 raft_consensus.cc:740] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.293322 10215 consensus_queue.cc:260] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.293789 10215 raft_consensus.cc:399] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.293964 10215 raft_consensus.cc:493] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.294190 10215 raft_consensus.cc:3060] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.295440 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:11.299021 10215 raft_consensus.cc:515] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.299398 10215 leader_election.cc:304] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.299820 10218 raft_consensus.cc:2804] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.300042 10218 raft_consensus.cc:697] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.300395 10218 consensus_queue.cc:237] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.301173 10215 leader_election.cc:290] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.301972 10215 ts_tablet_manager.cc:1434] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.011s user 0.002s sys 0.007s
I20260807 07:53:11.309603 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.012s sys 0.005s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":1205,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1183,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:11.310539 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 533 bytes on disk
I20260807 07:53:11.307870 9886 catalog_manager.cc:5697] T bc47f50c6ffe4a2c85c5b821e0910e8f P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.315001 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.315750 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.274523
I20260807 07:53:11.319681 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.011s sys 0.003s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":14091,"drs_written":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1363,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:11.328681 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 533 bytes on disk
I20260807 07:53:11.331234 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.332119 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.274090
I20260807 07:53:11.364151 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.030s sys 0.004s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":17657,"cfile_cache_miss":31,"cfile_cache_miss_bytes":26935,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1813,"dirs.run_cpu_time_us":573,"dirs.run_wall_time_us":2538,"drs_written":1,"lbm_read_time_us":1965,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1458,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":91,"update_count":2}
I20260807 07:53:11.365586 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4078 bytes on disk
I20260807 07:53:11.366410 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.373558 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.015s sys 0.023s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":17771,"cfile_cache_miss":31,"cfile_cache_miss_bytes":27105,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1002,"dirs.run_cpu_time_us":1072,"dirs.run_wall_time_us":5417,"drs_written":1,"lbm_read_time_us":1987,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1425,"lbm_writes_lt_1ms":31,"mutex_wait_us":174,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":91,"thread_start_us":2058,"threads_started":1,"update_count":2}
I20260807 07:53:11.374722 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4078 bytes on disk
I20260807 07:53:11.375463 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.408564 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36480:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.425084 10099 tablet_service.cc:1511] Processing CreateTablet for tablet b54b4817ae5c44e78cf8e71ac1514971 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.45 [id=14ba15f0b6574e07a87fe01a68fd3407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.425798 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54b4817ae5c44e78cf8e71ac1514971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.433430 10215 tablet_bootstrap.cc:492] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.435819 10215 tablet_bootstrap.cc:654] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.439749 10215 tablet_bootstrap.cc:492] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.440026 10215 ts_tablet_manager.cc:1403] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 07:53:11.441053 10215 raft_consensus.cc:359] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.441429 10215 raft_consensus.cc:385] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.441601 10215 raft_consensus.cc:740] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.442026 10215 consensus_queue.cc:260] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.442303 10215 raft_consensus.cc:399] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.442473 10215 raft_consensus.cc:493] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.442710 10215 raft_consensus.cc:3060] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.445271 10215 raft_consensus.cc:515] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.445568 10215 leader_election.cc:304] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.445835 10215 leader_election.cc:290] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.445914 10218 raft_consensus.cc:2804] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.446271 10218 raft_consensus.cc:697] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.446481 10215 ts_tablet_manager.cc:1434] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 07:53:11.446650 10218 consensus_queue.cc:237] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.452550 9886 catalog_manager.cc:5697] T b54b4817ae5c44e78cf8e71ac1514971 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.478135 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:11.498768 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.013s sys 0.004s Metrics: {"bytes_written":5627,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":252,"dirs.run_wall_time_us":1360,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1217,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"spinlock_wait_cycles":896}
I20260807 07:53:11.500206 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.261848
I20260807 07:53:11.509861 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36482:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.528224 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 6d61ccd3d9b94d928f105fe8ebc644a9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.46 [id=a32b5e55da404316a129d61d4d1bea19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.529008 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d61ccd3d9b94d928f105fe8ebc644a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.538988 10215 tablet_bootstrap.cc:492] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.541913 10215 tablet_bootstrap.cc:654] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.542415 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.026s sys 0.014s Metrics: {"bytes_written":25834,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18209,"cfile_cache_miss":31,"cfile_cache_miss_bytes":27709,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":695,"dirs.run_cpu_time_us":198,"dirs.run_wall_time_us":3432,"drs_written":1,"lbm_read_time_us":1713,"lbm_reads_lt_1ms":67,"lbm_write_time_us":2535,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":36,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":93,"spinlock_wait_cycles":5376}
I20260807 07:53:11.543649 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4186 bytes on disk
I20260807 07:53:11.544175 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.544797 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:11.546499 10215 tablet_bootstrap.cc:492] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.546734 10215 ts_tablet_manager.cc:1403] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260807 07:53:11.548110 10215 raft_consensus.cc:359] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.548381 10215 raft_consensus.cc:385] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.548612 10215 raft_consensus.cc:740] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.548982 10215 consensus_queue.cc:260] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.549322 10215 raft_consensus.cc:399] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.549541 10215 raft_consensus.cc:493] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.549754 10215 raft_consensus.cc:3060] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.552583 10215 raft_consensus.cc:515] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.552973 10215 leader_election.cc:304] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.553382 10215 leader_election.cc:290] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.553414 10218 raft_consensus.cc:2804] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.553596 10218 raft_consensus.cc:697] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.554018 10218 consensus_queue.cc:237] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.554250 10215 ts_tablet_manager.cc:1434] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.008s
I20260807 07:53:11.560761 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.007s sys 0.005s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1026,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":922,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:11.563122 9886 catalog_manager.cc:5697] T 6d61ccd3d9b94d928f105fe8ebc644a9 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.564049 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:11.564874 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":117,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.566021 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.184493
I20260807 07:53:11.600320 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.021s sys 0.011s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":18518,"cfile_cache_miss":28,"cfile_cache_miss_bytes":27540,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":912,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1258,"drs_written":1,"lbm_read_time_us":1129,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1413,"lbm_writes_lt_1ms":31,"mutex_wait_us":143,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":95,"spinlock_wait_cycles":39808,"update_count":1}
I20260807 07:53:11.601224 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4258 bytes on disk
I20260807 07:53:11.602197 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":164,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.617961 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260807 07:53:11.626866 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:11.628381 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36484:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.647665 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.011s sys 0.008s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1213,"drs_written":1,"lbm_read_time_us":171,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1426,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:11.649228 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 438 bytes on disk
I20260807 07:53:11.650919 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 41a6a00ad73f4c0daa3849e777e64637 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.47 [id=15f8ba52ac43432082b5fa59f538f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.651517 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a6a00ad73f4c0daa3849e777e64637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.651209 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.014s sys 0.005s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":1943,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1333,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1326,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:11.652329 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.652547 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259603
I20260807 07:53:11.653205 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259603
I20260807 07:53:11.662576 10215 tablet_bootstrap.cc:492] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.665731 10215 tablet_bootstrap.cc:654] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.678442 10215 tablet_bootstrap.cc:492] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.678740 10215 ts_tablet_manager.cc:1403] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.013s
I20260807 07:53:11.680272 10215 raft_consensus.cc:359] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.680621 10215 raft_consensus.cc:385] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.680929 10215 raft_consensus.cc:740] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.681540 10215 consensus_queue.cc:260] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.681792 10215 raft_consensus.cc:399] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.682021 10215 raft_consensus.cc:493] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.682202 10215 raft_consensus.cc:3060] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.686832 10215 raft_consensus.cc:515] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.687260 10215 leader_election.cc:304] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.687587 10218 raft_consensus.cc:2804] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.687836 10218 raft_consensus.cc:697] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.689324 10218 consensus_queue.cc:237] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.692864 10215 leader_election.cc:290] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.694088 10215 ts_tablet_manager.cc:1434] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.015s user 0.008s sys 0.002s
I20260807 07:53:11.698314 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.040s sys 0.004s Metrics: {"bytes_written":26263,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18540,"cfile_cache_miss":31,"cfile_cache_miss_bytes":28256,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":539,"dirs.run_cpu_time_us":264,"dirs.run_wall_time_us":1270,"drs_written":1,"lbm_read_time_us":1638,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1592,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":95}
I20260807 07:53:11.699568 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4269 bytes on disk
I20260807 07:53:11.700551 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.700403 9886 catalog_manager.cc:5697] T 41a6a00ad73f4c0daa3849e777e64637 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.701395 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260807 07:53:11.704110 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.033s sys 0.008s Metrics: {"bytes_written":26263,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18540,"cfile_cache_miss":31,"cfile_cache_miss_bytes":28256,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":344,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1072,"drs_written":1,"lbm_read_time_us":1755,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1658,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":95}
I20260807 07:53:11.705451 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4269 bytes on disk
I20260807 07:53:11.706293 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.706997 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000016
I20260807 07:53:11.726305 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.006s sys 0.013s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":146,"dirs.run_cpu_time_us":145,"dirs.run_wall_time_us":1555,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2092,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:11.726583 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.007s sys 0.011s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":82,"dirs.run_cpu_time_us":166,"dirs.run_wall_time_us":960,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1427,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:11.727450 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:11.727983 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:11.738209 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.003s sys 0.006s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":193,"lbm_read_time_us":372,"lbm_reads_lt_1ms":25,"lbm_write_time_us":290,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:11.738860 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:11.739542 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.740195 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223531
I20260807 07:53:11.741490 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.007s sys 0.003s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":182,"lbm_read_time_us":345,"lbm_reads_lt_1ms":21,"lbm_write_time_us":392,"lbm_writes_lt_1ms":9,"mutex_wait_us":24,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:11.742301 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.222491
I20260807 07:53:11.770022 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.024s sys 0.004s Metrics: {"bytes_written":26711,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18951,"cfile_cache_miss":31,"cfile_cache_miss_bytes":28529,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":964,"dirs.run_cpu_time_us":1151,"dirs.run_wall_time_us":5499,"drs_written":1,"lbm_read_time_us":943,"lbm_reads_lt_1ms":63,"lbm_write_time_us":933,"lbm_writes_lt_1ms":27,"mutex_wait_us":61,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":97,"spinlock_wait_cycles":3712,"thread_start_us":771,"threads_started":1}
I20260807 07:53:11.770911 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4374 bytes on disk
I20260807 07:53:11.771584 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.776542 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.032s sys 0.000s Metrics: {"bytes_written":26711,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18951,"cfile_cache_miss":31,"cfile_cache_miss_bytes":28529,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1844,"dirs.run_cpu_time_us":1123,"dirs.run_wall_time_us":6549,"drs_written":1,"lbm_read_time_us":1415,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1211,"lbm_writes_lt_1ms":27,"mutex_wait_us":305,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":97,"spinlock_wait_cycles":3456,"thread_start_us":712,"threads_started":1}
I20260807 07:53:11.777398 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4374 bytes on disk
I20260807 07:53:11.778616 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260807 07:53:11.797987 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36490:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.811841 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 556d8f6bf7a24cdd8bdf005a1a7a11b1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.48 [id=ae83ddc98e9f454780faaedcb0f42e05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.812393 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 556d8f6bf7a24cdd8bdf005a1a7a11b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.818589 10215 tablet_bootstrap.cc:492] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.821317 10215 tablet_bootstrap.cc:654] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.824924 10215 tablet_bootstrap.cc:492] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.825129 10215 ts_tablet_manager.cc:1403] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 07:53:11.826294 10215 raft_consensus.cc:359] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.826478 10215 raft_consensus.cc:385] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.826625 10215 raft_consensus.cc:740] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.826875 10215 consensus_queue.cc:260] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.827147 10215 raft_consensus.cc:399] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.827251 10215 raft_consensus.cc:493] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.827445 10215 raft_consensus.cc:3060] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.829944 10215 raft_consensus.cc:515] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.830327 10215 leader_election.cc:304] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.830662 10215 leader_election.cc:290] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.830735 10218 raft_consensus.cc:2804] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.830895 10218 raft_consensus.cc:697] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.831204 10218 consensus_queue.cc:237] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.831487 10215 ts_tablet_manager.cc:1434] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 07:53:11.837193 9886 catalog_manager.cc:5697] T 556d8f6bf7a24cdd8bdf005a1a7a11b1 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.853375 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:11.869527 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.011s sys 0.005s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":260,"dirs.run_wall_time_us":1226,"drs_written":1,"lbm_read_time_us":137,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1014,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:11.870620 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.232172
I20260807 07:53:11.899348 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36498:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:11.913312 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.028s sys 0.007s Metrics: {"bytes_written":27136,"cfile_cache_hit":13,"cfile_cache_hit_bytes":19280,"cfile_cache_miss":31,"cfile_cache_miss_bytes":29314,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1503,"dirs.run_cpu_time_us":550,"dirs.run_wall_time_us":2453,"drs_written":1,"lbm_read_time_us":2301,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1321,"lbm_writes_lt_1ms":27,"mutex_wait_us":160,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":99,"spinlock_wait_cycles":1408}
I20260807 07:53:11.915496 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:11.916395 10099 tablet_service.cc:1511] Processing CreateTablet for tablet c3f4b524f98a485a9e766eeb4ce117f1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.49 [id=6edc536184cf477a992caab970561de7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:11.917018 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3f4b524f98a485a9e766eeb4ce117f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:11.926146 10215 tablet_bootstrap.cc:492] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:11.928782 10215 tablet_bootstrap.cc:654] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:11.931078 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.003s sys 0.010s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":1014,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1278,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:11.931912 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4688 bytes on disk
I20260807 07:53:11.932608 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":99,"lbm_reads_lt_1ms":8}
I20260807 07:53:11.933265 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235637
I20260807 07:53:11.945951 10215 tablet_bootstrap.cc:492] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:11.946200 10215 ts_tablet_manager.cc:1403] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.004s
I20260807 07:53:11.947458 10215 raft_consensus.cc:359] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.947790 10215 raft_consensus.cc:385] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:11.948001 10215 raft_consensus.cc:740] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:11.948534 10215 consensus_queue.cc:260] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.948778 10215 raft_consensus.cc:399] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:11.948980 10215 raft_consensus.cc:493] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:11.949158 10215 raft_consensus.cc:3060] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:11.953493 10215 raft_consensus.cc:515] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.953934 10215 leader_election.cc:304] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:11.954320 10218 raft_consensus.cc:2804] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:11.954340 10215 leader_election.cc:290] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:11.954668 10218 raft_consensus.cc:697] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:11.955607 10218 consensus_queue.cc:237] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:11.955907 10215 ts_tablet_manager.cc:1434] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260807 07:53:11.963994 9886 catalog_manager.cc:5697] T c3f4b524f98a485a9e766eeb4ce117f1 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:11.964174 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.021s sys 0.008s Metrics: {"bytes_written":27545,"cfile_cache_hit":12,"cfile_cache_hit_bytes":19623,"cfile_cache_miss":28,"cfile_cache_miss_bytes":29177,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2198,"dirs.run_cpu_time_us":938,"dirs.run_wall_time_us":4870,"drs_written":1,"lbm_read_time_us":978,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1115,"lbm_writes_lt_1ms":27,"mutex_wait_us":49,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":101,"spinlock_wait_cycles":6528}
I20260807 07:53:11.965376 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4525 bytes on disk
I20260807 07:53:11.967298 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.022782 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:12.025117 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36500:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:12.030109 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:12.041555 10099 tablet_service.cc:1511] Processing CreateTablet for tablet fd505eb9d6754117abed4411bcee55d1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.50 [id=2b717dcd410843429ae50394a4c3b881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:12.042239 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd505eb9d6754117abed4411bcee55d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:12.045145 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.008s sys 0.008s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":4454,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1778,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1705,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:12.046299 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.255931
I20260807 07:53:12.048604 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.008s sys 0.006s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":243,"dirs.run_wall_time_us":1050,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1291,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:12.049937 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 472 bytes on disk
I20260807 07:53:12.050757 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.051501 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.255931
I20260807 07:53:12.051828 10215 tablet_bootstrap.cc:492] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:12.054728 10215 tablet_bootstrap.cc:654] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:12.059244 10215 tablet_bootstrap.cc:492] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:12.059417 10215 ts_tablet_manager.cc:1403] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260807 07:53:12.060835 10215 raft_consensus.cc:359] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.061143 10215 raft_consensus.cc:385] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:12.061509 10215 raft_consensus.cc:740] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:12.061853 10215 consensus_queue.cc:260] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.062143 10215 raft_consensus.cc:399] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:12.062278 10215 raft_consensus.cc:493] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:12.062539 10215 raft_consensus.cc:3060] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:12.066975 10215 raft_consensus.cc:515] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.067412 10215 leader_election.cc:304] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:12.067726 10215 leader_election.cc:290] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:12.067919 10218 raft_consensus.cc:2804] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:12.068205 10218 raft_consensus.cc:697] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:12.068738 10218 consensus_queue.cc:237] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.068900 10215 ts_tablet_manager.cc:1434] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260807 07:53:12.078146 9886 catalog_manager.cc:5697] T fd505eb9d6754117abed4411bcee55d1 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:12.084370 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.031s sys 0.000s Metrics: {"bytes_written":27582,"cfile_cache_hit":12,"cfile_cache_hit_bytes":19658,"cfile_cache_miss":28,"cfile_cache_miss_bytes":29532,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":848,"dirs.run_cpu_time_us":782,"dirs.run_wall_time_us":4282,"drs_written":1,"lbm_read_time_us":1263,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1213,"lbm_writes_lt_1ms":27,"mutex_wait_us":204,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":101,"spinlock_wait_cycles":896}
I20260807 07:53:12.085568 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4556 bytes on disk
I20260807 07:53:12.086297 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":126,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.087024 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:12.099488 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.027s sys 0.010s Metrics: {"bytes_written":27582,"cfile_cache_hit":12,"cfile_cache_hit_bytes":19658,"cfile_cache_miss":28,"cfile_cache_miss_bytes":29532,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":921,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":6078,"drs_written":1,"lbm_read_time_us":2362,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1534,"lbm_writes_lt_1ms":27,"mutex_wait_us":32,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":101,"spinlock_wait_cycles":2944}
I20260807 07:53:12.100544 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4556 bytes on disk
I20260807 07:53:12.104300 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.105055 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000017
I20260807 07:53:12.106931 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.012s sys 0.004s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":1077,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1497,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:12.107805 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260807 07:53:12.122480 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.014s sys 0.002s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1066,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1170,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.123864 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:12.124743 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":159,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.125468 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.244296
I20260807 07:53:12.132824 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.009s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":3367,"lbm_read_time_us":581,"lbm_reads_lt_1ms":25,"lbm_write_time_us":609,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:12.133605 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249502
I20260807 07:53:12.166307 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.033s sys 0.004s Metrics: {"bytes_written":28015,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20036,"cfile_cache_miss":28,"cfile_cache_miss_bytes":30046,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1141,"dirs.run_cpu_time_us":679,"dirs.run_wall_time_us":3321,"drs_written":1,"lbm_read_time_us":1435,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1371,"lbm_writes_lt_1ms":27,"mutex_wait_us":709,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":103,"spinlock_wait_cycles":1030016}
I20260807 07:53:12.167399 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4643 bytes on disk
I20260807 07:53:12.168269 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.176476 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36504:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:12.179782 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.027s sys 0.011s Metrics: {"bytes_written":28015,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20069,"cfile_cache_miss":31,"cfile_cache_miss_bytes":30225,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1170,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":3182,"drs_written":1,"lbm_read_time_us":1614,"lbm_reads_lt_1ms":67,"lbm_write_time_us":3998,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":31,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":103}
I20260807 07:53:12.181322 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4643 bytes on disk
I20260807 07:53:12.183321 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.192808 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd3da4d168c495d900bc8388ddab135 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.51 [id=7478205bb4dd47c394654ee0138d4f04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:12.193390 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd3da4d168c495d900bc8388ddab135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:12.199173 10215 tablet_bootstrap.cc:492] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:12.201318 10215 tablet_bootstrap.cc:654] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:12.204818 10215 tablet_bootstrap.cc:492] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:12.204963 10215 ts_tablet_manager.cc:1403] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:12.205760 10215 raft_consensus.cc:359] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.206079 10215 raft_consensus.cc:385] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:12.206228 10215 raft_consensus.cc:740] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:12.206529 10215 consensus_queue.cc:260] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.206785 10215 raft_consensus.cc:399] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:12.206984 10215 raft_consensus.cc:493] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:12.207176 10215 raft_consensus.cc:3060] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:12.209895 10215 raft_consensus.cc:515] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.210166 10215 leader_election.cc:304] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:12.210518 10218 raft_consensus.cc:2804] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:12.210537 10215 leader_election.cc:290] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:12.210829 10218 raft_consensus.cc:697] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:12.211323 10218 consensus_queue.cc:237] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.211441 10215 ts_tablet_manager.cc:1434] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 07:53:12.217253 9887 catalog_manager.cc:5697] T 2fd3da4d168c495d900bc8388ddab135 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:12.218422 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:12.242610 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.008s sys 0.006s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":270,"dirs.run_wall_time_us":1312,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2006,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":4,"update_count":1}
I20260807 07:53:12.243616 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 426 bytes on disk
I20260807 07:53:12.244174 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.244853 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.310313
I20260807 07:53:12.254190 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":879,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":281,"lbm_read_time_us":382,"lbm_reads_lt_1ms":21,"lbm_write_time_us":288,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:12.255020 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.265824
I20260807 07:53:12.320094 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.047s sys 0.007s Metrics: {"bytes_written":28472,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20395,"cfile_cache_miss":31,"cfile_cache_miss_bytes":30867,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1153,"dirs.run_cpu_time_us":1566,"dirs.run_wall_time_us":5494,"drs_written":1,"lbm_read_time_us":2169,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1480,"lbm_writes_lt_1ms":27,"mutex_wait_us":86,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":105,"spinlock_wait_cycles":13568}
I20260807 07:53:12.322130 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4724 bytes on disk
I20260807 07:53:12.323729 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.336812 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36506:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:12.365813 10099 tablet_service.cc:1511] Processing CreateTablet for tablet ba52b3aa24524bb8be89565addb39558 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.52 [id=15a2687c648d439685e5f4aad0204c88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:12.366632 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba52b3aa24524bb8be89565addb39558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:12.378751 10215 tablet_bootstrap.cc:492] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:12.383610 10215 tablet_bootstrap.cc:654] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:12.390174 10215 tablet_bootstrap.cc:492] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:12.390552 10215 ts_tablet_manager.cc:1403] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260807 07:53:12.393038 10215 raft_consensus.cc:359] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.393589 10215 raft_consensus.cc:385] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:12.393923 10215 raft_consensus.cc:740] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:12.394464 10215 consensus_queue.cc:260] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.394846 10215 raft_consensus.cc:399] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:12.395009 10215 raft_consensus.cc:493] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:12.395283 10215 raft_consensus.cc:3060] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:12.400414 10215 raft_consensus.cc:515] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.400946 10215 leader_election.cc:304] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:12.401569 10215 leader_election.cc:290] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:12.401844 10218 raft_consensus.cc:2804] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:12.402259 10218 raft_consensus.cc:697] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:12.402959 10218 consensus_queue.cc:237] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.403654 10215 ts_tablet_manager.cc:1434] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260807 07:53:12.419878 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260807 07:53:12.435307 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000091
I20260807 07:53:12.459745 9886 catalog_manager.cc:5697] T ba52b3aa24524bb8be89565addb39558 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:12.469739 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.027s sys 0.010s Metrics: {"bytes_written":5591,"cfile_init":1,"dirs.queue_time_us":225,"dirs.run_cpu_time_us":242,"dirs.run_wall_time_us":1289,"drs_written":1,"lbm_read_time_us":205,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3186,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:12.473166 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 426 bytes on disk
I20260807 07:53:12.474151 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.020s sys 0.015s Metrics: {"bytes_written":5591,"cfile_init":1,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":945,"drs_written":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3008,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:12.474568 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":151,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.475937 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.253469
I20260807 07:53:12.475903 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.253469
I20260807 07:53:12.568446 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.091s user 0.059s sys 0.024s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20726,"cfile_cache_miss":28,"cfile_cache_miss_bytes":31044,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":3681,"dirs.run_cpu_time_us":711,"dirs.run_wall_time_us":2373,"drs_written":1,"lbm_read_time_us":2963,"lbm_reads_lt_1ms":56,"lbm_write_time_us":3718,"lbm_writes_lt_1ms":31,"mutex_wait_us":1492,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":107,"update_count":1}
I20260807 07:53:12.573302 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4788 bytes on disk
I20260807 07:53:12.576334 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000093
I20260807 07:53:12.576334 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33768:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:12.576960 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":220,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.601183 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.124s user 0.050s sys 0.035s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20726,"cfile_cache_miss":28,"cfile_cache_miss_bytes":31044,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":5387,"dirs.run_cpu_time_us":1878,"dirs.run_wall_time_us":8099,"drs_written":1,"lbm_read_time_us":3897,"lbm_reads_lt_1ms":60,"lbm_write_time_us":9608,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":1206,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":107,"update_count":1}
I20260807 07:53:12.610574 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4788 bytes on disk
I20260807 07:53:12.612488 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":222,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.613864 10099 tablet_service.cc:1511] Processing CreateTablet for tablet eb67d825c20f48a7a00ceed54723da57 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.53 [id=c1f2722d5c8d49539623f860e757c3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:12.614998 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb67d825c20f48a7a00ceed54723da57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:12.615958 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000039
I20260807 07:53:12.633533 10215 tablet_bootstrap.cc:492] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:12.634042 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.055s user 0.024s sys 0.012s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":106,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1032,"drs_written":1,"lbm_read_time_us":273,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3667,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.636843 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.241360
I20260807 07:53:12.644279 10215 tablet_bootstrap.cc:654] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:12.657245 10215 tablet_bootstrap.cc:492] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:12.658228 10215 ts_tablet_manager.cc:1403] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.010s
I20260807 07:53:12.658622 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.016s sys 0.019s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":6881,"dirs.run_cpu_time_us":328,"dirs.run_wall_time_us":1345,"drs_written":1,"lbm_read_time_us":260,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2247,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.661484 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:12.661952 10215 raft_consensus.cc:359] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.662752 10215 raft_consensus.cc:385] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:12.663177 10215 raft_consensus.cc:740] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:12.664368 10215 consensus_queue.cc:260] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.664819 10215 raft_consensus.cc:399] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:12.665566 10215 raft_consensus.cc:493] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:12.666123 10215 raft_consensus.cc:3060] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:12.665751 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.000s sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":1969,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260807 07:53:12.668031 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207291
I20260807 07:53:12.674748 10215 raft_consensus.cc:515] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.675326 10215 leader_election.cc:304] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:12.676493 10215 leader_election.cc:290] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:12.676826 10218 raft_consensus.cc:2804] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:12.678016 10218 raft_consensus.cc:697] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:12.678551 10215 ts_tablet_manager.cc:1434] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
I20260807 07:53:12.678989 10218 consensus_queue.cc:237] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.698283 9886 catalog_manager.cc:5697] T eb67d825c20f48a7a00ceed54723da57 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:12.732549 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.095s user 0.051s sys 0.026s Metrics: {"bytes_written":28896,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20776,"cfile_cache_miss":28,"cfile_cache_miss_bytes":31140,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":3555,"dirs.run_cpu_time_us":1715,"dirs.run_wall_time_us":8711,"drs_written":1,"lbm_read_time_us":3489,"lbm_reads_lt_1ms":60,"lbm_write_time_us":2918,"lbm_writes_lt_1ms":27,"mutex_wait_us":1389,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":107}
I20260807 07:53:12.734686 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000028
I20260807 07:53:12.771864 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.023s sys 0.010s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":116,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":1322,"drs_written":1,"lbm_read_time_us":151,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2785,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.774266 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.212431
I20260807 07:53:12.787791 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.119s user 0.072s sys 0.016s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21100,"cfile_cache_miss":31,"cfile_cache_miss_bytes":31748,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":929,"dirs.run_cpu_time_us":1835,"dirs.run_wall_time_us":5569,"drs_written":1,"lbm_read_time_us":3662,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3464,"lbm_writes_lt_1ms":31,"mutex_wait_us":23,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":109,"thread_start_us":11021,"threads_started":1,"update_count":1}
I20260807 07:53:12.791098 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4868 bytes on disk
I20260807 07:53:12.792681 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":202,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.826339 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33780:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:12.829514 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000100
I20260807 07:53:12.860831 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 802228e5f531445fa71d3f271af22933 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.54 [id=1b012496e7b84ca89a54f9b11b7d28e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:12.861881 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802228e5f531445fa71d3f271af22933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:12.875350 10215 tablet_bootstrap.cc:492] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:12.876962 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.102s user 0.055s sys 0.028s Metrics: {"bytes_written":29339,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21141,"cfile_cache_miss":28,"cfile_cache_miss_bytes":31659,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":2204,"dirs.run_cpu_time_us":1237,"dirs.run_wall_time_us":7438,"drs_written":1,"lbm_read_time_us":10418,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":61,"lbm_write_time_us":2643,"lbm_writes_lt_1ms":27,"mutex_wait_us":232,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":109,"spinlock_wait_cycles":35200,"thread_start_us":4742,"threads_started":1}
I20260807 07:53:12.879077 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000030
I20260807 07:53:12.880127 10215 tablet_bootstrap.cc:654] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:12.882083 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.021s sys 0.013s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":103,"dirs.run_cpu_time_us":265,"dirs.run_wall_time_us":1246,"drs_written":1,"lbm_read_time_us":261,"lbm_reads_lt_1ms":4,"lbm_write_time_us":9947,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.884510 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.212067
I20260807 07:53:12.889776 10215 tablet_bootstrap.cc:492] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:12.890110 10215 ts_tablet_manager.cc:1403] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260807 07:53:12.892006 10215 raft_consensus.cc:359] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.892478 10215 raft_consensus.cc:385] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:12.892867 10215 raft_consensus.cc:740] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:12.893438 10215 consensus_queue.cc:260] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.893927 10215 raft_consensus.cc:399] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:12.894169 10215 raft_consensus.cc:493] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:12.894403 10215 raft_consensus.cc:3060] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:12.898309 10215 raft_consensus.cc:515] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.898736 10215 leader_election.cc:304] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:12.899318 10215 leader_election.cc:290] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:12.899427 10218 raft_consensus.cc:2804] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:12.899718 10218 raft_consensus.cc:697] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:12.900211 10218 consensus_queue.cc:237] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:12.902705 10215 ts_tablet_manager.cc:1434] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260807 07:53:12.921670 9886 catalog_manager.cc:5697] T 802228e5f531445fa71d3f271af22933 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:12.924906 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.023s sys 0.010s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":345,"dirs.run_cpu_time_us":268,"dirs.run_wall_time_us":1298,"drs_written":1,"lbm_read_time_us":161,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2720,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:12.926937 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.209909
I20260807 07:53:12.973796 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.088s user 0.058s sys 0.017s Metrics: {"bytes_written":29339,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21138,"cfile_cache_miss":31,"cfile_cache_miss_bytes":32030,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1323,"dirs.run_cpu_time_us":1436,"dirs.run_wall_time_us":5017,"drs_written":1,"lbm_read_time_us":3869,"lbm_reads_lt_1ms":67,"lbm_write_time_us":3165,"lbm_writes_lt_1ms":27,"mutex_wait_us":457,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":109,"spinlock_wait_cycles":1408}
I20260807 07:53:12.975868 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4884 bytes on disk
I20260807 07:53:12.977432 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":198,"lbm_reads_lt_1ms":4}
I20260807 07:53:12.979164 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000027
I20260807 07:53:13.005213 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.077s user 0.049s sys 0.019s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21478,"cfile_cache_miss":28,"cfile_cache_miss_bytes":31916,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":2691,"dirs.run_cpu_time_us":1367,"dirs.run_wall_time_us":4745,"drs_written":1,"lbm_read_time_us":2960,"lbm_reads_lt_1ms":64,"lbm_write_time_us":3076,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":111,"thread_start_us":1222,"threads_started":1,"update_count":2}
I20260807 07:53:13.006747 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4948 bytes on disk
I20260807 07:53:13.007053 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.013s sys 0.013s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":105,"dirs.run_cpu_time_us":307,"dirs.run_wall_time_us":1304,"drs_written":1,"lbm_read_time_us":132,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2054,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:13.008337 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":235,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.009281 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.442536
I20260807 07:53:13.033068 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.014s sys 0.005s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":613,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":293,"lbm_read_time_us":927,"lbm_reads_lt_1ms":25,"lbm_write_time_us":932,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:13.034797 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:13.036182 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.038185 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224220
I20260807 07:53:13.045128 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000107
I20260807 07:53:13.104719 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33790:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:13.115072 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.069s user 0.023s sys 0.013s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":1000,"dirs.run_cpu_time_us":333,"dirs.run_wall_time_us":1826,"drs_written":1,"lbm_read_time_us":211,"lbm_reads_lt_1ms":4,"lbm_write_time_us":23027,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.117702 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 384 bytes on disk
I20260807 07:53:13.119511 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":192,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.121701 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.215012
I20260807 07:53:13.146838 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a207a375916343a4beaac84350d90c0d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.55 [id=ffbcd26c9d504d32a3b055beeff1b0f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:13.147807 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a207a375916343a4beaac84350d90c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:13.152119 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.113s user 0.060s sys 0.022s Metrics: {"bytes_written":29768,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21550,"cfile_cache_miss":31,"cfile_cache_miss_bytes":32626,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":5157,"dirs.run_cpu_time_us":1456,"dirs.run_wall_time_us":5867,"drs_written":1,"lbm_read_time_us":11261,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":9338,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":2374,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":111}
I20260807 07:53:13.154594 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000041
I20260807 07:53:13.166941 10215 tablet_bootstrap.cc:492] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:13.174273 10215 tablet_bootstrap.cc:654] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:13.190814 10215 tablet_bootstrap.cc:492] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:13.191532 10215 ts_tablet_manager.cc:1403] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.008s
I20260807 07:53:13.194468 10215 raft_consensus.cc:359] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.198098 10215 raft_consensus.cc:385] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:13.198398 10215 raft_consensus.cc:740] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:13.199394 10215 consensus_queue.cc:260] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.200048 10215 raft_consensus.cc:399] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:13.200531 10215 raft_consensus.cc:493] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:13.200937 10215 raft_consensus.cc:3060] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:13.211295 10215 raft_consensus.cc:515] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.212190 10215 leader_election.cc:304] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:13.213056 10218 raft_consensus.cc:2804] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:13.213583 10218 raft_consensus.cc:697] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:13.213471 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.091s user 0.050s sys 0.023s Metrics: {"bytes_written":29768,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21513,"cfile_cache_miss":31,"cfile_cache_miss_bytes":32617,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":653,"dirs.run_cpu_time_us":1090,"dirs.run_wall_time_us":4768,"drs_written":1,"lbm_read_time_us":3406,"lbm_reads_lt_1ms":63,"lbm_write_time_us":2660,"lbm_writes_lt_1ms":27,"mutex_wait_us":697,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":111}
I20260807 07:53:13.214511 10218 consensus_queue.cc:237] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.215971 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000028
I20260807 07:53:13.222342 10215 leader_election.cc:290] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:13.224458 10215 ts_tablet_manager.cc:1434] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.033s user 0.017s sys 0.001s
I20260807 07:53:13.235723 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.076s user 0.023s sys 0.013s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":91,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1200,"drs_written":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4399,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.237748 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5212 bytes on disk
I20260807 07:53:13.240486 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":650,"lbm_reads_lt_1ms":8}
I20260807 07:53:13.243045 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.202119
I20260807 07:53:13.260401 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000093
I20260807 07:53:13.264593 9887 catalog_manager.cc:5697] T a207a375916343a4beaac84350d90c0d P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:13.294536 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.078s user 0.022s sys 0.011s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":103,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1038,"drs_written":1,"lbm_read_time_us":158,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2483,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.306447 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.202119
I20260807 07:53:13.371995 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.110s user 0.028s sys 0.014s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":125,"dirs.run_cpu_time_us":390,"dirs.run_wall_time_us":16791,"drs_written":1,"lbm_read_time_us":246,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2742,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:13.375169 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:13.377864 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.124s user 0.083s sys 0.032s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21843,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32467,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":5099,"dirs.run_cpu_time_us":1734,"dirs.run_wall_time_us":16335,"drs_written":1,"lbm_read_time_us":3163,"lbm_reads_lt_1ms":56,"lbm_write_time_us":4110,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":113,"thread_start_us":31182,"threads_started":1,"update_count":1}
I20260807 07:53:13.380618 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000005
I20260807 07:53:13.395481 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.007s sys 0.003s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":757,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:13.396554 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:13.410051 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.102s user 0.052s sys 0.030s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":21843,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32467,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1810,"dirs.run_cpu_time_us":1927,"dirs.run_wall_time_us":7082,"drs_written":1,"lbm_read_time_us":3445,"lbm_reads_lt_1ms":64,"lbm_write_time_us":8337,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":1387,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":113,"update_count":1}
I20260807 07:53:13.412343 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5065 bytes on disk
I20260807 07:53:13.420606 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.009s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":85,"lbm_read_time_us":437,"lbm_reads_lt_1ms":8,"lbm_write_time_us":370,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:13.421932 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5048 bytes on disk
I20260807 07:53:13.423084 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":234,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.424739 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":289,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.429412 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.017s sys 0.003s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":191,"lbm_read_time_us":1159,"lbm_reads_lt_1ms":21,"lbm_write_time_us":3230,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"mutex_wait_us":3,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:13.431282 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.211304
I20260807 07:53:13.485450 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33792:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:13.520910 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 1a0fb7f140b0418a897fb831550af8b0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.56 [id=af6ec9682e96499e8eae6f872b3eef45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:13.522111 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a0fb7f140b0418a897fb831550af8b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:13.522318 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.090s user 0.065s sys 0.016s Metrics: {"bytes_written":30215,"cfile_cache_hit":14,"cfile_cache_hit_bytes":21908,"cfile_cache_miss":34,"cfile_cache_miss_bytes":33374,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1938,"dirs.run_cpu_time_us":2058,"dirs.run_wall_time_us":6882,"drs_written":1,"lbm_read_time_us":2609,"lbm_reads_lt_1ms":74,"lbm_write_time_us":2480,"lbm_writes_lt_1ms":27,"mutex_wait_us":49,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":113,"spinlock_wait_cycles":3840,"thread_start_us":3157,"threads_started":2}
I20260807 07:53:13.524003 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5082 bytes on disk
I20260807 07:53:13.525883 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":273,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.527309 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000043
I20260807 07:53:13.536412 10215 tablet_bootstrap.cc:492] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:13.542992 10215 tablet_bootstrap.cc:654] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:13.550669 10215 tablet_bootstrap.cc:492] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:13.551349 10215 ts_tablet_manager.cc:1403] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260807 07:53:13.554481 10215 raft_consensus.cc:359] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.554844 10215 raft_consensus.cc:385] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:13.555306 10215 raft_consensus.cc:740] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:13.555799 10215 consensus_queue.cc:260] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.556233 10215 raft_consensus.cc:399] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:13.556478 10215 raft_consensus.cc:493] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:13.556742 10215 raft_consensus.cc:3060] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:13.564416 10215 raft_consensus.cc:515] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.563890 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.023s sys 0.008s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":107,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1204,"drs_written":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2602,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.565060 10215 leader_election.cc:304] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:13.565799 10215 leader_election.cc:290] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:13.566124 10219 raft_consensus.cc:2804] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:13.566516 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.175564
I20260807 07:53:13.566671 10219 raft_consensus.cc:697] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:13.567584 10215 ts_tablet_manager.cc:1434] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260807 07:53:13.567502 10219 consensus_queue.cc:237] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.590167 9886 catalog_manager.cc:5697] T 1a0fb7f140b0418a897fb831550af8b0 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:13.653123 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.086s user 0.058s sys 0.023s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":22218,"cfile_cache_miss":28,"cfile_cache_miss_bytes":33054,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1954,"dirs.run_cpu_time_us":1640,"dirs.run_wall_time_us":6050,"drs_written":1,"lbm_read_time_us":3453,"lbm_reads_lt_1ms":60,"lbm_write_time_us":3619,"lbm_writes_lt_1ms":31,"mutex_wait_us":603,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":115,"spinlock_wait_cycles":25856,"thread_start_us":1353,"threads_started":1,"update_count":1}
I20260807 07:53:13.654979 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5151 bytes on disk
I20260807 07:53:13.656293 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":126,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.676067 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000122
I20260807 07:53:13.677443 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000106
I20260807 07:53:13.717268 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33794:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:13.729645 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.024s sys 0.011s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":155,"dirs.run_cpu_time_us":439,"dirs.run_wall_time_us":1588,"drs_written":1,"lbm_read_time_us":309,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2787,"lbm_writes_lt_1ms":27,"mutex_wait_us":46,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.731709 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.179953
I20260807 07:53:13.744446 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.026s sys 0.010s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":1049,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":3989,"drs_written":1,"lbm_read_time_us":209,"lbm_reads_lt_1ms":4,"lbm_write_time_us":14988,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.746440 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.179951
I20260807 07:53:13.760473 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 4f82c335d1ad484096c444474ff7da4e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.57 [id=cf45ad1201aa4b2bac779404f92dc06f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:13.761574 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f82c335d1ad484096c444474ff7da4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:13.775068 10215 tablet_bootstrap.cc:492] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:13.781286 10215 tablet_bootstrap.cc:654] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:13.788213 10215 tablet_bootstrap.cc:492] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:13.788589 10215 ts_tablet_manager.cc:1403] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260807 07:53:13.790628 10215 raft_consensus.cc:359] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.791095 10215 raft_consensus.cc:385] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:13.791353 10215 raft_consensus.cc:740] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:13.792063 10215 consensus_queue.cc:260] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.792523 10215 raft_consensus.cc:399] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:13.792862 10215 raft_consensus.cc:493] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:13.793181 10215 raft_consensus.cc:3060] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:13.799640 10215 raft_consensus.cc:515] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.800081 10215 leader_election.cc:304] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:13.800833 10215 leader_election.cc:290] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:13.800833 10219 raft_consensus.cc:2804] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:13.801532 10219 raft_consensus.cc:697] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:13.802359 10219 consensus_queue.cc:237] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:13.810312 10215 ts_tablet_manager.cc:1434] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.021s user 0.017s sys 0.004s
W20260807 07:53:13.814473 10180 tablet.cc:2406] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 07:53:13.827781 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.095s user 0.069s sys 0.023s Metrics: {"bytes_written":30640,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22287,"cfile_cache_miss":31,"cfile_cache_miss_bytes":33737,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1760,"dirs.run_cpu_time_us":1722,"dirs.run_wall_time_us":7351,"drs_written":1,"lbm_read_time_us":3352,"lbm_reads_lt_1ms":67,"lbm_write_time_us":3244,"lbm_writes_lt_1ms":27,"mutex_wait_us":162,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":115,"spinlock_wait_cycles":217856,"thread_start_us":14551,"threads_started":1}
I20260807 07:53:13.831393 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5165 bytes on disk
I20260807 07:53:13.832690 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":144,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.834383 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000029
I20260807 07:53:13.839963 9886 catalog_manager.cc:5697] T 4f82c335d1ad484096c444474ff7da4e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:13.869087 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.122s user 0.060s sys 0.031s Metrics: {"bytes_written":30640,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22253,"cfile_cache_miss":31,"cfile_cache_miss_bytes":33775,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1637,"dirs.run_cpu_time_us":1490,"dirs.run_wall_time_us":5281,"drs_written":1,"lbm_read_time_us":3883,"lbm_reads_lt_1ms":67,"lbm_write_time_us":3023,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":115,"spinlock_wait_cycles":2304,"thread_start_us":12404,"threads_started":2}
I20260807 07:53:13.870810 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000035
I20260807 07:53:13.880267 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.016s sys 0.017s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":88,"dirs.run_cpu_time_us":297,"dirs.run_wall_time_us":1125,"drs_written":1,"lbm_read_time_us":385,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4045,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:13.883996 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:13.885406 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260807 07:53:13.887899 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:13.903564 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.018s sys 0.010s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":101,"dirs.run_cpu_time_us":292,"dirs.run_wall_time_us":1240,"drs_written":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2912,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:13.905455 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.001s sys 0.009s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"lbm_read_time_us":541,"lbm_reads_lt_1ms":17,"lbm_write_time_us":459,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:13.905779 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.442536
I20260807 07:53:13.906486 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.185349
I20260807 07:53:13.908087 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000096
I20260807 07:53:13.935045 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.019s sys 0.008s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":613,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":79,"lbm_read_time_us":1620,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1040,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:13.937212 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.189937
I20260807 07:53:13.943738 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.021s sys 0.008s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":271,"dirs.run_wall_time_us":1323,"drs_written":1,"lbm_read_time_us":354,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2301,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:13.944941 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.180863
I20260807 07:53:14.007803 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.101s user 0.071s sys 0.024s Metrics: {"bytes_written":31086,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22651,"cfile_cache_miss":31,"cfile_cache_miss_bytes":34033,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1868,"dirs.run_cpu_time_us":1500,"dirs.run_wall_time_us":5072,"drs_written":1,"lbm_read_time_us":3575,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3589,"lbm_writes_lt_1ms":27,"mutex_wait_us":65,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":117,"spinlock_wait_cycles":17920,"thread_start_us":8352,"threads_started":1}
I20260807 07:53:14.010396 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5266 bytes on disk
I20260807 07:53:14.014279 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":272,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.052608 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.113s user 0.070s sys 0.035s Metrics: {"bytes_written":31086,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22655,"cfile_cache_miss":31,"cfile_cache_miss_bytes":34285,"cfile_init":10,"delta_iterators_relevant":6,"dirs.queue_time_us":3085,"dirs.run_cpu_time_us":2054,"dirs.run_wall_time_us":6377,"drs_written":1,"lbm_read_time_us":4531,"lbm_reads_lt_1ms":71,"lbm_write_time_us":3435,"lbm_writes_lt_1ms":27,"mutex_wait_us":35,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":117,"spinlock_wait_cycles":2560,"thread_start_us":18686,"threads_started":1}
I20260807 07:53:14.054605 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5266 bytes on disk
I20260807 07:53:14.056666 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":304,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.062381 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.117s user 0.063s sys 0.044s Metrics: {"bytes_written":31086,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22618,"cfile_cache_miss":31,"cfile_cache_miss_bytes":34286,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":6285,"dirs.run_cpu_time_us":1638,"dirs.run_wall_time_us":8959,"drs_written":1,"lbm_read_time_us":3875,"lbm_reads_lt_1ms":67,"lbm_write_time_us":2949,"lbm_writes_lt_1ms":27,"mutex_wait_us":226,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":117,"thread_start_us":17872,"threads_started":1}
I20260807 07:53:14.064945 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5266 bytes on disk
I20260807 07:53:14.067371 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":286,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.074440 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33804:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:14.103603 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 054e50c8462840509f2c06265c9670b6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.58 [id=4c8d56f72d7647828728bb52e8d1c031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:14.104904 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054e50c8462840509f2c06265c9670b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:14.117854 10215 tablet_bootstrap.cc:492] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:14.125173 10215 tablet_bootstrap.cc:654] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:14.132709 10215 tablet_bootstrap.cc:492] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:14.133179 10215 ts_tablet_manager.cc:1403] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.002s
I20260807 07:53:14.136066 10215 raft_consensus.cc:359] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.136478 10215 raft_consensus.cc:385] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:14.136945 10215 raft_consensus.cc:740] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:14.137753 10215 consensus_queue.cc:260] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.138306 10215 raft_consensus.cc:399] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:14.138559 10215 raft_consensus.cc:493] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:14.138895 10215 raft_consensus.cc:3060] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:14.146664 10215 raft_consensus.cc:515] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.147383 10215 leader_election.cc:304] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:14.147913 10215 leader_election.cc:290] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:14.148488 10219 raft_consensus.cc:2804] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:14.149084 10219 raft_consensus.cc:697] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:14.149896 10215 ts_tablet_manager.cc:1434] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.016s user 0.009s sys 0.006s
I20260807 07:53:14.150665 10219 consensus_queue.cc:237] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.183125 9886 catalog_manager.cc:5697] T 054e50c8462840509f2c06265c9670b6 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:14.272485 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000109
I20260807 07:53:14.309322 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000113
I20260807 07:53:14.315270 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33814:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:14.319020 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000104
I20260807 07:53:14.355557 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 40499d9d1bfb407bb92c3a4f7aa225c9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.59 [id=dcdce98f783449f685c67fff59453ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:14.357111 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40499d9d1bfb407bb92c3a4f7aa225c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:14.399019 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.087s user 0.028s sys 0.013s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":698,"dirs.run_cpu_time_us":367,"dirs.run_wall_time_us":1469,"drs_written":1,"lbm_read_time_us":282,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2904,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.402590 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:14.404817 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":370,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.406788 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.205814
I20260807 07:53:14.411823 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.092s user 0.016s sys 0.013s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":85,"dirs.run_cpu_time_us":269,"dirs.run_wall_time_us":1490,"drs_written":1,"lbm_read_time_us":266,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10247,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.413558 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:14.416144 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":160,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.418522 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.205814
I20260807 07:53:14.426872 10215 tablet_bootstrap.cc:492] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:14.438446 10215 tablet_bootstrap.cc:654] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:14.447568 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.174s user 0.022s sys 0.017s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":145,"dirs.run_cpu_time_us":340,"dirs.run_wall_time_us":1586,"drs_written":1,"lbm_read_time_us":330,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3322,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.449738 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:14.453779 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":300,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.463392 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.205814
I20260807 07:53:14.471190 10215 tablet_bootstrap.cc:492] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:14.471962 10215 ts_tablet_manager.cc:1403] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.008s
I20260807 07:53:14.475776 10215 raft_consensus.cc:359] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.488734 10215 raft_consensus.cc:385] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:14.489006 10215 raft_consensus.cc:740] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:14.489773 10215 consensus_queue.cc:260] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.490391 10219 raft_consensus.cc:493] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 07:53:14.491082 10219 raft_consensus.cc:515] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.492209 10219 leader_election.cc:304] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:14.490432 10215 raft_consensus.cc:399] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:14.493212 10215 raft_consensus.cc:493] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:14.493498 10215 raft_consensus.cc:3060] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:14.497697 10219 leader_election.cc:290] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260807 07:53:14.510267 10215 raft_consensus.cc:515] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.513866 10215 leader_election.cc:304] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:14.514461 10215 leader_election.cc:290] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:14.514140 10891 raft_consensus.cc:2764] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 07:53:14.529986 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.122s user 0.064s sys 0.029s Metrics: {"bytes_written":31517,"cfile_cache_hit":12,"cfile_cache_hit_bytes":22996,"cfile_cache_miss":28,"cfile_cache_miss_bytes":34440,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":759,"dirs.run_cpu_time_us":1411,"dirs.run_wall_time_us":4962,"drs_written":1,"lbm_read_time_us":6922,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":8154,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":119}
I20260807 07:53:14.531283 10219 raft_consensus.cc:2804] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:14.532195 10219 raft_consensus.cc:697] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:14.533118 10219 consensus_queue.cc:237] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.534047 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5352 bytes on disk
I20260807 07:53:14.539760 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.542440 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000040
I20260807 07:53:14.591835 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.015s sys 0.013s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":114,"dirs.run_cpu_time_us":350,"dirs.run_wall_time_us":1262,"drs_written":1,"lbm_read_time_us":228,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3038,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.598788 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.174080
I20260807 07:53:14.599282 9887 catalog_manager.cc:5697] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:14.605397 10215 ts_tablet_manager.cc:1434] T 40499d9d1bfb407bb92c3a4f7aa225c9 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.133s user 0.018s sys 0.022s
I20260807 07:53:14.691850 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.228s user 0.071s sys 0.048s Metrics: {"bytes_written":31517,"cfile_cache_hit":12,"cfile_cache_hit_bytes":22996,"cfile_cache_miss":28,"cfile_cache_miss_bytes":34440,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":5411,"dirs.run_cpu_time_us":1735,"dirs.run_wall_time_us":6645,"drs_written":1,"lbm_read_time_us":11854,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":2998,"lbm_writes_lt_1ms":27,"mutex_wait_us":359,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":119,"thread_start_us":35225,"threads_started":2}
I20260807 07:53:14.696589 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000069
I20260807 07:53:14.698607 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.279s user 0.075s sys 0.027s Metrics: {"bytes_written":31517,"cfile_cache_hit":12,"cfile_cache_hit_bytes":22996,"cfile_cache_miss":28,"cfile_cache_miss_bytes":34440,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2855,"dirs.run_cpu_time_us":1545,"dirs.run_wall_time_us":5378,"drs_written":1,"lbm_read_time_us":12654,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":53,"lbm_write_time_us":20158,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":148,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":119,"thread_start_us":74662,"threads_started":2}
I20260807 07:53:14.700627 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:14.754436 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.029s sys 0.000s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":261,"dirs.run_wall_time_us":1169,"drs_written":1,"lbm_read_time_us":171,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2620,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.756512 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.179311
I20260807 07:53:14.775233 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.176s user 0.077s sys 0.021s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23323,"cfile_cache_miss":28,"cfile_cache_miss_bytes":34695,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1259,"dirs.run_cpu_time_us":2620,"dirs.run_wall_time_us":7192,"drs_written":1,"lbm_read_time_us":25307,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":58,"lbm_write_time_us":3810,"lbm_writes_lt_1ms":31,"mutex_wait_us":567,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":121,"update_count":2}
I20260807 07:53:14.778103 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5425 bytes on disk
I20260807 07:53:14.776525 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.072s user 0.028s sys 0.004s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":86,"dirs.run_cpu_time_us":375,"dirs.run_wall_time_us":1548,"drs_written":1,"lbm_read_time_us":313,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2681,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.780036 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":396,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.781191 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5735 bytes on disk
I20260807 07:53:14.785925 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":469,"lbm_reads_lt_1ms":8}
I20260807 07:53:14.788393 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.179311
I20260807 07:53:14.872373 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33818:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:14.903952 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.110s user 0.060s sys 0.035s Metrics: {"bytes_written":31964,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23361,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35025,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2639,"dirs.run_cpu_time_us":1399,"dirs.run_wall_time_us":6999,"drs_written":1,"lbm_read_time_us":3157,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2808,"lbm_writes_lt_1ms":27,"mutex_wait_us":1617,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":121,"spinlock_wait_cycles":41856,"thread_start_us":22367,"threads_started":1}
I20260807 07:53:14.906450 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000036
I20260807 07:53:14.915931 10099 tablet_service.cc:1511] Processing CreateTablet for tablet efdb51c4278b47b28e421c02819c64b3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.60 [id=a230aec35eec48ccbb1a915049b6089b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:14.916963 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efdb51c4278b47b28e421c02819c64b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:14.918833 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.157s user 0.057s sys 0.019s Metrics: {"bytes_written":31964,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23361,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35025,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":178,"dirs.run_cpu_time_us":299,"dirs.run_wall_time_us":1287,"drs_written":1,"lbm_read_time_us":6039,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3052,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":121,"spinlock_wait_cycles":13056}
I20260807 07:53:14.922107 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5452 bytes on disk
I20260807 07:53:14.925618 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":279,"lbm_reads_lt_1ms":4}
I20260807 07:53:14.929703 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000049
I20260807 07:53:14.947984 10215 tablet_bootstrap.cc:492] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:14.955758 10215 tablet_bootstrap.cc:654] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:14.961601 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.031s sys 0.006s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":95,"dirs.run_cpu_time_us":299,"dirs.run_wall_time_us":1221,"drs_written":1,"lbm_read_time_us":211,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6549,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.964073 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5687 bytes on disk
I20260807 07:53:14.964059 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.030s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":5952,"dirs.run_cpu_time_us":361,"dirs.run_wall_time_us":1279,"drs_written":1,"lbm_read_time_us":137,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2891,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:14.966117 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.205593
I20260807 07:53:14.966675 10215 tablet_bootstrap.cc:492] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:14.966909 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":586,"lbm_reads_lt_1ms":8}
I20260807 07:53:14.967330 10215 ts_tablet_manager.cc:1403] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.005s
I20260807 07:53:14.969076 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.205593
I20260807 07:53:14.969956 10215 raft_consensus.cc:359] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.970486 10215 raft_consensus.cc:385] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:14.970798 10215 raft_consensus.cc:740] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:14.971438 10215 consensus_queue.cc:260] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.972045 10215 raft_consensus.cc:399] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:14.972373 10215 raft_consensus.cc:493] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:14.972709 10215 raft_consensus.cc:3060] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:14.981599 10215 raft_consensus.cc:515] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.982609 10215 leader_election.cc:304] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:14.983536 10930 raft_consensus.cc:2804] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:14.983932 10930 raft_consensus.cc:697] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:14.984612 10930 consensus_queue.cc:237] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:14.991891 10215 leader_election.cc:290] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:14.994652 10215 ts_tablet_manager.cc:1434] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.026s user 0.011s sys 0.005s
I20260807 07:53:15.035259 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000123
I20260807 07:53:15.043313 9887 catalog_manager.cc:5697] T efdb51c4278b47b28e421c02819c64b3 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:15.112016 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.076s user 0.027s sys 0.012s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":114,"dirs.run_cpu_time_us":269,"dirs.run_wall_time_us":1303,"drs_written":1,"lbm_read_time_us":217,"lbm_reads_lt_1ms":4,"lbm_write_time_us":20572,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:15.114066 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.144s user 0.077s sys 0.019s Metrics: {"bytes_written":32357,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23698,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35280,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2986,"dirs.run_cpu_time_us":1627,"dirs.run_wall_time_us":19767,"drs_written":1,"lbm_read_time_us":3590,"lbm_reads_lt_1ms":56,"lbm_write_time_us":3205,"lbm_writes_lt_1ms":27,"mutex_wait_us":52,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":123,"spinlock_wait_cycles":2944,"thread_start_us":13761,"threads_started":1}
I20260807 07:53:15.115206 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:15.117053 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":268,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.118170 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000012
I20260807 07:53:15.120774 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:15.142305 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.012s sys 0.002s Metrics: {"bytes_written":390,"delete_count":0,"lbm_write_time_us":476,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:15.143898 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5518 bytes on disk
I20260807 07:53:15.144711 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.177s user 0.073s sys 0.027s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23698,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35280,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1123,"dirs.run_cpu_time_us":1588,"dirs.run_wall_time_us":5559,"drs_written":1,"lbm_read_time_us":5525,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":59,"lbm_write_time_us":4633,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":123,"spinlock_wait_cycles":768,"thread_start_us":7253,"threads_started":1,"update_count":2}
I20260807 07:53:15.146142 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":329,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.147595 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5518 bytes on disk
I20260807 07:53:15.148934 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":181,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.149169 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.014s sys 0.011s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":397,"lbm_read_time_us":821,"lbm_reads_lt_1ms":17,"lbm_write_time_us":828,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:15.150725 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.214065
I20260807 07:53:15.257536 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.106s user 0.067s sys 0.028s Metrics: {"bytes_written":32394,"cfile_cache_hit":14,"cfile_cache_hit_bytes":23763,"cfile_cache_miss":34,"cfile_cache_miss_bytes":36191,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":927,"dirs.run_cpu_time_us":1931,"dirs.run_wall_time_us":7290,"drs_written":1,"lbm_read_time_us":4528,"lbm_reads_lt_1ms":70,"lbm_write_time_us":3288,"lbm_writes_lt_1ms":27,"mutex_wait_us":181,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":123}
I20260807 07:53:15.259420 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5552 bytes on disk
I20260807 07:53:15.260918 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":312,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.303782 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33826:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:15.341387 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a1c9d0c3aaba48bf94de39fa195a170f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.61 [id=23f95290de574bc493d0f30a0e1d2e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:15.342085 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1c9d0c3aaba48bf94de39fa195a170f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:15.354612 10215 tablet_bootstrap.cc:492] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:15.358722 10215 tablet_bootstrap.cc:654] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:15.362836 10215 tablet_bootstrap.cc:492] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:15.363193 10215 ts_tablet_manager.cc:1403] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 07:53:15.365015 10215 raft_consensus.cc:359] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.365494 10215 raft_consensus.cc:385] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:15.365654 10215 raft_consensus.cc:740] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:15.366303 10215 consensus_queue.cc:260] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.366662 10215 raft_consensus.cc:399] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:15.366883 10215 raft_consensus.cc:493] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:15.367086 10215 raft_consensus.cc:3060] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:15.373399 10215 raft_consensus.cc:515] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.373893 10215 leader_election.cc:304] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:15.374323 10215 leader_election.cc:290] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:15.374409 10219 raft_consensus.cc:2804] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:15.376493 10215 ts_tablet_manager.cc:1434] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 07:53:15.377321 10219 raft_consensus.cc:697] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:15.377823 10219 consensus_queue.cc:237] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.394049 9886 catalog_manager.cc:5697] T a1c9d0c3aaba48bf94de39fa195a170f P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:15.398285 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000121
I20260807 07:53:15.408475 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000124
I20260807 07:53:15.451712 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.029s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":269,"dirs.run_wall_time_us":1420,"drs_written":1,"lbm_read_time_us":211,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2461,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:15.454241 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:15.470907 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":236,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.472919 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231036
I20260807 07:53:15.480264 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.070s user 0.019s sys 0.008s Metrics: {"bytes_written":335,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":1467,"dirs.run_cpu_time_us":286,"dirs.run_wall_time_us":1336,"drs_written":1,"lbm_read_time_us":201,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2628,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:15.485077 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.463347
I20260807 07:53:15.512912 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000111
I20260807 07:53:15.555164 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.081s user 0.060s sys 0.017s Metrics: {"bytes_written":32836,"cfile_cache_hit":13,"cfile_cache_hit_bytes":24060,"cfile_cache_miss":31,"cfile_cache_miss_bytes":36280,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1941,"dirs.run_cpu_time_us":1381,"dirs.run_wall_time_us":4716,"drs_written":1,"lbm_read_time_us":3658,"lbm_reads_lt_1ms":63,"lbm_write_time_us":2254,"lbm_writes_lt_1ms":27,"mutex_wait_us":1130,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":125,"spinlock_wait_cycles":26368}
I20260807 07:53:15.557482 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5634 bytes on disk
I20260807 07:53:15.559883 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.070s user 0.016s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":579,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":6671,"dirs.run_cpu_time_us":469,"dirs.run_wall_time_us":2108,"lbm_read_time_us":1034,"lbm_reads_lt_1ms":25,"lbm_write_time_us":705,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:15.561017 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:15.563117 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":257,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.564962 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.240262
I20260807 07:53:15.566184 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":366,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.576906 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.024s sys 0.004s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":10918,"dirs.run_cpu_time_us":445,"dirs.run_wall_time_us":1471,"drs_written":1,"lbm_read_time_us":230,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1854,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:15.579224 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:15.580742 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":244,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.582703 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235414
I20260807 07:53:15.614898 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33840:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:15.674528 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.091s user 0.058s sys 0.025s Metrics: {"bytes_written":32836,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24101,"cfile_cache_miss":28,"cfile_cache_miss_bytes":36123,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":3217,"dirs.run_cpu_time_us":1472,"dirs.run_wall_time_us":9792,"drs_written":1,"lbm_read_time_us":3486,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2786,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":125,"thread_start_us":7390,"threads_started":1}
I20260807 07:53:15.676651 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000028
I20260807 07:53:15.676862 10099 tablet_service.cc:1511] Processing CreateTablet for tablet bf2388abdbf3423b8f01d5faaedeeca0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.62 [id=2284387bffff46c89d070f0ab687ae71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:15.678138 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2388abdbf3423b8f01d5faaedeeca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:15.693848 10215 tablet_bootstrap.cc:492] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:15.700910 10215 tablet_bootstrap.cc:654] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:15.710574 10215 tablet_bootstrap.cc:492] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:15.711033 10215 ts_tablet_manager.cc:1403] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.013s
I20260807 07:53:15.713536 10215 raft_consensus.cc:359] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.713431 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.147s user 0.079s sys 0.023s Metrics: {"bytes_written":32836,"cfile_cache_hit":14,"cfile_cache_hit_bytes":24131,"cfile_cache_miss":34,"cfile_cache_miss_bytes":36741,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":5633,"dirs.run_cpu_time_us":1859,"dirs.run_wall_time_us":8703,"drs_written":1,"lbm_read_time_us":3757,"lbm_reads_lt_1ms":70,"lbm_write_time_us":3384,"lbm_writes_lt_1ms":27,"mutex_wait_us":2759,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":125,"thread_start_us":28817,"threads_started":2}
I20260807 07:53:15.714067 10215 raft_consensus.cc:385] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:15.714537 10215 raft_consensus.cc:740] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:15.715184 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000065
I20260807 07:53:15.715229 10215 consensus_queue.cc:260] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.718103 10215 raft_consensus.cc:399] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:15.718317 10215 raft_consensus.cc:493] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:15.718655 10215 raft_consensus.cc:3060] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:15.724177 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.024s sys 0.002s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":113,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1144,"drs_written":1,"lbm_read_time_us":235,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3023,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:15.725219 10215 raft_consensus.cc:515] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.725942 10215 leader_election.cc:304] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:15.726477 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5869 bytes on disk
I20260807 07:53:15.726686 10219 raft_consensus.cc:2804] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:15.727114 10219 raft_consensus.cc:697] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:15.727674 10219 consensus_queue.cc:237] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:15.728569 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":436,"lbm_reads_lt_1ms":8}
I20260807 07:53:15.730247 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231662
I20260807 07:53:15.732640 10215 leader_election.cc:290] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:15.735090 10215 ts_tablet_manager.cc:1434] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.024s user 0.007s sys 0.006s
I20260807 07:53:15.743697 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.012s sys 0.010s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":632,"dirs.run_cpu_time_us":253,"dirs.run_wall_time_us":1257,"drs_written":1,"lbm_read_time_us":247,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4442,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:15.745487 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5869 bytes on disk
I20260807 07:53:15.747658 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":382,"lbm_reads_lt_1ms":8}
I20260807 07:53:15.749814 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231662
I20260807 07:53:15.776011 9887 catalog_manager.cc:5697] T bf2388abdbf3423b8f01d5faaedeeca0 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:15.818286 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000102
I20260807 07:53:15.863648 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.022s sys 0.009s Metrics: {"bytes_written":5192,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":274,"dirs.run_wall_time_us":1176,"drs_written":1,"lbm_read_time_us":256,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3342,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:15.865942 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.235640
I20260807 07:53:15.898779 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.159s user 0.061s sys 0.014s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24438,"cfile_cache_miss":28,"cfile_cache_miss_bytes":36382,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":10907,"dirs.run_cpu_time_us":1540,"dirs.run_wall_time_us":5729,"drs_written":1,"lbm_read_time_us":36374,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":2845,"lbm_writes_lt_1ms":32,"mutex_wait_us":50,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":127,"thread_start_us":6304,"threads_started":1,"update_count":2}
I20260807 07:53:15.900749 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5698 bytes on disk
I20260807 07:53:15.902617 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":307,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.912376 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.162s user 0.093s sys 0.031s Metrics: {"bytes_written":390,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24438,"cfile_cache_miss":28,"cfile_cache_miss_bytes":36382,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1428,"dirs.run_cpu_time_us":1436,"dirs.run_wall_time_us":6762,"drs_written":1,"lbm_read_time_us":3483,"lbm_reads_lt_1ms":56,"lbm_write_time_us":2990,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":127,"thread_start_us":40378,"threads_started":1,"update_count":2}
I20260807 07:53:15.914417 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5698 bytes on disk
I20260807 07:53:15.916122 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":176,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.974051 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33854:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:15.987443 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.117s user 0.063s sys 0.026s Metrics: {"bytes_written":33289,"cfile_cache_hit":12,"cfile_cache_hit_bytes":24473,"cfile_cache_miss":28,"cfile_cache_miss_bytes":36661,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1708,"dirs.run_cpu_time_us":1658,"dirs.run_wall_time_us":8253,"drs_written":1,"lbm_read_time_us":3229,"lbm_reads_lt_1ms":60,"lbm_write_time_us":3050,"lbm_writes_lt_1ms":28,"mutex_wait_us":88,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":127,"thread_start_us":7007,"threads_started":1}
I20260807 07:53:15.990978 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5721 bytes on disk
I20260807 07:53:15.992900 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":235,"lbm_reads_lt_1ms":4}
I20260807 07:53:15.994323 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000036
I20260807 07:53:16.011705 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f976a2e7c8064ba59ebe3d5fd106db04 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.63 [id=812018ca55084cba97d3fd7f9fdd877a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:16.013072 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f976a2e7c8064ba59ebe3d5fd106db04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:16.024065 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.017s sys 0.010s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":92,"dirs.run_cpu_time_us":293,"dirs.run_wall_time_us":1740,"drs_written":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2848,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:16.026469 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260807 07:53:16.027848 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":166,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.029742 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:16.030683 10215 tablet_bootstrap.cc:492] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:16.037052 10215 tablet_bootstrap.cc:654] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:16.044464 10215 tablet_bootstrap.cc:492] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:16.045011 10215 ts_tablet_manager.cc:1403] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260807 07:53:16.047233 10215 raft_consensus.cc:359] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.047725 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.011s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":156,"lbm_read_time_us":617,"lbm_reads_lt_1ms":21,"lbm_write_time_us":664,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:16.048167 10215 raft_consensus.cc:385] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:16.048727 10215 raft_consensus.cc:740] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:16.049029 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.233657
I20260807 07:53:16.049703 10215 consensus_queue.cc:260] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.050326 10215 raft_consensus.cc:399] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:16.050824 10215 raft_consensus.cc:493] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:16.051321 10215 raft_consensus.cc:3060] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:16.057847 10215 raft_consensus.cc:515] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.058406 10215 leader_election.cc:304] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:16.059145 10215 leader_election.cc:290] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:16.059295 10219 raft_consensus.cc:2804] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:16.059779 10219 raft_consensus.cc:697] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:16.060281 10219 consensus_queue.cc:237] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.064826 10215 ts_tablet_manager.cc:1434] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.019s user 0.002s sys 0.017s
I20260807 07:53:16.083273 9887 catalog_manager.cc:5697] T f976a2e7c8064ba59ebe3d5fd106db04 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:16.138870 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.089s user 0.068s sys 0.014s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":24836,"cfile_cache_miss":31,"cfile_cache_miss_bytes":36990,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":2973,"dirs.run_cpu_time_us":2257,"dirs.run_wall_time_us":7871,"drs_written":1,"lbm_read_time_us":3341,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3884,"lbm_writes_lt_1ms":32,"mutex_wait_us":117,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":129,"spinlock_wait_cycles":295936,"thread_start_us":1462,"threads_started":1,"update_count":1}
I20260807 07:53:16.140151 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260807 07:53:16.146361 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":250,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:16.147665 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5792 bytes on disk
I20260807 07:53:16.148713 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.149869 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:16.154608 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000120
I20260807 07:53:16.167734 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000118
I20260807 07:53:16.166721 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":72,"dirs.run_cpu_time_us":272,"dirs.run_wall_time_us":779,"lbm_read_time_us":332,"lbm_reads_lt_1ms":8,"lbm_write_time_us":354,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:16.184296 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.018s sys 0.007s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":91,"dirs.run_cpu_time_us":296,"dirs.run_wall_time_us":1175,"drs_written":1,"lbm_read_time_us":189,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2080,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.188750 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.234294
I20260807 07:53:16.203066 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.024s sys 0.004s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1306,"drs_written":1,"lbm_read_time_us":326,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2200,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.205207 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.234294
I20260807 07:53:16.274978 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33870:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:16.355372 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.166s user 0.118s sys 0.011s Metrics: {"bytes_written":33733,"cfile_cache_hit":13,"cfile_cache_hit_bytes":24838,"cfile_cache_miss":31,"cfile_cache_miss_bytes":37714,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2663,"dirs.run_cpu_time_us":329,"dirs.run_wall_time_us":5627,"drs_written":1,"lbm_read_time_us":3767,"lbm_reads_lt_1ms":67,"lbm_write_time_us":2887,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":129}
I20260807 07:53:16.382740 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5820 bytes on disk
I20260807 07:53:16.424708 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000110
I20260807 07:53:16.478020 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.272s user 0.076s sys 0.058s Metrics: {"bytes_written":33733,"cfile_cache_hit":13,"cfile_cache_hit_bytes":24838,"cfile_cache_miss":31,"cfile_cache_miss_bytes":37714,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":9767,"dirs.run_cpu_time_us":1773,"dirs.run_wall_time_us":38726,"drs_written":1,"lbm_read_time_us":3576,"lbm_reads_lt_1ms":67,"lbm_write_time_us":2938,"lbm_writes_lt_1ms":28,"mutex_wait_us":3666,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":129,"thread_start_us":194023,"threads_started":1}
I20260807 07:53:16.479866 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260807 07:53:16.468969 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":2867,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260807 07:53:16.487566 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:16.493706 10099 tablet_service.cc:1511] Processing CreateTablet for tablet c9dfd3b6e6fb48bdb12783101d818995 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.64 [id=6f2d301699454e309d57c1b0415c471e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:16.494809 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9dfd3b6e6fb48bdb12783101d818995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:16.520756 10215 tablet_bootstrap.cc:492] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:16.529289 10215 tablet_bootstrap.cc:654] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:16.529699 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.024s sys 0.002s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":8331,"dirs.run_cpu_time_us":306,"dirs.run_wall_time_us":1330,"drs_written":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4,"lbm_write_time_us":5101,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:16.530730 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.098s user 0.041s sys 0.006s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":1302,"dirs.run_cpu_time_us":326,"dirs.run_wall_time_us":1640,"drs_written":1,"lbm_read_time_us":209,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2638,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:16.531440 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5987 bytes on disk
I20260807 07:53:16.533107 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260807 07:53:16.534436 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":368,"lbm_reads_lt_1ms":8}
I20260807 07:53:16.534818 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":242,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.535769 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:16.536660 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:16.538455 10215 tablet_bootstrap.cc:492] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:16.538720 10215 ts_tablet_manager.cc:1403] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20260807 07:53:16.540788 10215 raft_consensus.cc:359] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.541239 10215 raft_consensus.cc:385] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:16.541664 10215 raft_consensus.cc:740] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:16.542321 10215 consensus_queue.cc:260] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.542918 10215 raft_consensus.cc:399] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:16.543143 10215 raft_consensus.cc:493] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:16.543953 10215 raft_consensus.cc:3060] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:16.544888 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.022s sys 0.009s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":305,"dirs.run_cpu_time_us":335,"dirs.run_wall_time_us":1325,"drs_written":1,"lbm_read_time_us":266,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2537,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.546466 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.228872
I20260807 07:53:16.548871 10215 raft_consensus.cc:515] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.549293 10215 leader_election.cc:304] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:16.549939 10219 raft_consensus.cc:2804] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:16.550352 10219 raft_consensus.cc:697] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:16.550963 10219 consensus_queue.cc:237] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.556324 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.017s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":124,"lbm_read_time_us":1039,"lbm_reads_lt_1ms":21,"lbm_write_time_us":768,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:16.557153 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.232505
I20260807 07:53:16.557868 10215 leader_election.cc:290] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:16.559517 10215 ts_tablet_manager.cc:1434] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.021s user 0.011s sys 0.000s
I20260807 07:53:16.559629 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.012s sys 0.008s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":174,"dirs.run_cpu_time_us":358,"dirs.run_wall_time_us":1540,"lbm_read_time_us":1000,"lbm_reads_lt_1ms":21,"lbm_write_time_us":811,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:16.560974 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.232224
I20260807 07:53:16.605223 9887 catalog_manager.cc:5697] T c9dfd3b6e6fb48bdb12783101d818995 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:16.663751 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.106s user 0.066s sys 0.006s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25211,"cfile_cache_miss":31,"cfile_cache_miss_bytes":37551,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1854,"dirs.run_cpu_time_us":1722,"dirs.run_wall_time_us":7115,"drs_written":1,"lbm_read_time_us":3500,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3307,"lbm_writes_lt_1ms":32,"mutex_wait_us":1489,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":131,"update_count":1}
I20260807 07:53:16.665485 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5895 bytes on disk
I20260807 07:53:16.666714 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":236,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.673494 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.126s user 0.058s sys 0.062s Metrics: {"bytes_written":34121,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25178,"cfile_cache_miss":28,"cfile_cache_miss_bytes":37482,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":10383,"dirs.run_cpu_time_us":1597,"dirs.run_wall_time_us":9119,"drs_written":1,"lbm_read_time_us":3212,"lbm_reads_lt_1ms":60,"lbm_write_time_us":2531,"lbm_writes_lt_1ms":28,"mutex_wait_us":2665,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":131,"thread_start_us":49938,"threads_started":1}
I20260807 07:53:16.676132 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000014
I20260807 07:53:16.678395 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.117s user 0.075s sys 0.012s Metrics: {"bytes_written":334,"cfile_cache_hit":14,"cfile_cache_hit_bytes":25208,"cfile_cache_miss":34,"cfile_cache_miss_bytes":38002,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":7894,"dirs.run_cpu_time_us":1898,"dirs.run_wall_time_us":8620,"drs_written":1,"lbm_read_time_us":3945,"lbm_reads_lt_1ms":70,"lbm_write_time_us":3063,"lbm_writes_lt_1ms":32,"mutex_wait_us":172,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":131,"thread_start_us":9876,"threads_started":2,"update_count":1}
I20260807 07:53:16.680634 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5895 bytes on disk
I20260807 07:53:16.682217 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":220,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.692507 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.002s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":417,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:16.693869 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5895 bytes on disk
I20260807 07:53:16.695767 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":228,"lbm_reads_lt_1ms":4}
I20260807 07:53:16.800772 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33884:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:16.827913 10099 tablet_service.cc:1511] Processing CreateTablet for tablet e73ae8e53cde4280aa7430c91e26e41b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.65 [id=ea9241b532f54f97958713911c974504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:16.828872 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73ae8e53cde4280aa7430c91e26e41b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:16.839289 10215 tablet_bootstrap.cc:492] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:16.843755 10215 tablet_bootstrap.cc:654] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:16.849635 10215 tablet_bootstrap.cc:492] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:16.850099 10215 ts_tablet_manager.cc:1403] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260807 07:53:16.852254 10215 raft_consensus.cc:359] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.852772 10215 raft_consensus.cc:385] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:16.852975 10215 raft_consensus.cc:740] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:16.853504 10215 consensus_queue.cc:260] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.853796 10215 raft_consensus.cc:399] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:16.854188 10215 raft_consensus.cc:493] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:16.854555 10215 raft_consensus.cc:3060] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:16.859134 10215 raft_consensus.cc:515] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.859623 10215 leader_election.cc:304] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:16.860023 10215 leader_election.cc:290] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:16.860455 11009 raft_consensus.cc:2804] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:16.861040 11009 raft_consensus.cc:697] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:16.861524 10215 ts_tablet_manager.cc:1434] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260807 07:53:16.862005 11009 consensus_queue.cc:237] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:16.876303 9887 catalog_manager.cc:5697] T e73ae8e53cde4280aa7430c91e26e41b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:16.918586 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000108
I20260807 07:53:16.934197 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000112
I20260807 07:53:16.948078 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000112
I20260807 07:53:16.949544 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.011s sys 0.019s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":92,"dirs.run_cpu_time_us":360,"dirs.run_wall_time_us":1520,"drs_written":1,"lbm_read_time_us":185,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2303,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.951303 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.204812
I20260807 07:53:16.969508 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.014s sys 0.014s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":104,"dirs.run_cpu_time_us":397,"dirs.run_wall_time_us":1182,"drs_written":1,"lbm_read_time_us":181,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3050,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.971894 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.204812
I20260807 07:53:16.989917 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.025s sys 0.007s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":133,"dirs.run_cpu_time_us":278,"dirs.run_wall_time_us":1244,"drs_written":1,"lbm_read_time_us":260,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2824,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:16.992129 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.204812
I20260807 07:53:17.054618 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33890:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.080063 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.107s user 0.068s sys 0.014s Metrics: {"bytes_written":34609,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25578,"cfile_cache_miss":31,"cfile_cache_miss_bytes":38666,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":198,"dirs.run_cpu_time_us":466,"dirs.run_wall_time_us":1580,"drs_written":1,"lbm_read_time_us":3765,"lbm_reads_lt_1ms":63,"lbm_write_time_us":4243,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":133}
I20260807 07:53:17.081945 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6007 bytes on disk
I20260807 07:53:17.083161 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":188,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.084259 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000032
I20260807 07:53:17.104791 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 536f4f07ab934f55bde5be14733c2683 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.66 [id=dba8fe12ec884b09939740547c035a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.106079 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536f4f07ab934f55bde5be14733c2683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.123751 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.018s sys 0.011s Metrics: {"bytes_written":5044,"cfile_init":1,"dirs.queue_time_us":2710,"dirs.run_cpu_time_us":321,"dirs.run_wall_time_us":989,"drs_written":1,"lbm_read_time_us":193,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3210,"lbm_writes_lt_1ms":27,"mutex_wait_us":2605,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.126586 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 167 bytes on disk
I20260807 07:53:17.125198 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.173s user 0.081s sys 0.012s Metrics: {"bytes_written":34609,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25578,"cfile_cache_miss":31,"cfile_cache_miss_bytes":38666,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":900,"dirs.run_cpu_time_us":1601,"dirs.run_wall_time_us":11970,"drs_written":1,"lbm_read_time_us":26713,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":61,"lbm_write_time_us":12467,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":93,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":133}
I20260807 07:53:17.128731 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6007 bytes on disk
I20260807 07:53:17.130088 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":445,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.131191 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000050
I20260807 07:53:17.138365 10215 tablet_bootstrap.cc:492] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.138335 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.139364 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.167205
I20260807 07:53:17.140939 10215 tablet_bootstrap.cc:654] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.149726 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.013s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":165,"dirs.run_cpu_time_us":309,"dirs.run_wall_time_us":1233,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1486,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.150703 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:17.152616 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.153192 10215 tablet_bootstrap.cc:492] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.153249 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.171909
I20260807 07:53:17.153460 10215 ts_tablet_manager.cc:1403] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260807 07:53:17.157569 10215 raft_consensus.cc:359] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.157904 10215 raft_consensus.cc:385] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.158154 10215 raft_consensus.cc:740] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.158560 10215 consensus_queue.cc:260] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.158855 10215 raft_consensus.cc:399] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.159004 10215 raft_consensus.cc:493] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.159878 10215 raft_consensus.cc:3060] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.166126 10215 raft_consensus.cc:515] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.166555 10215 leader_election.cc:304] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.166924 11009 raft_consensus.cc:2804] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.167127 11009 raft_consensus.cc:697] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.167392 11009 consensus_queue.cc:237] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.174868 10215 leader_election.cc:290] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.175900 10215 ts_tablet_manager.cc:1434] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.022s user 0.003s sys 0.006s
I20260807 07:53:17.189287 9887 catalog_manager.cc:5697] T 536f4f07ab934f55bde5be14733c2683 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.192292 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.039s sys 0.004s Metrics: {"bytes_written":34997,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25804,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38356,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":4689,"dirs.run_cpu_time_us":704,"dirs.run_wall_time_us":3634,"drs_written":1,"lbm_read_time_us":1367,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1703,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":135,"thread_start_us":7755,"threads_started":1}
I20260807 07:53:17.195138 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6072 bytes on disk
I20260807 07:53:17.195925 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.200374 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.037s sys 0.006s Metrics: {"bytes_written":34997,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25918,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38594,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":856,"dirs.run_cpu_time_us":573,"dirs.run_wall_time_us":3158,"drs_written":1,"lbm_read_time_us":1326,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1791,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":135,"thread_start_us":4999,"threads_started":1}
I20260807 07:53:17.202723 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6072 bytes on disk
I20260807 07:53:17.203501 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":123,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.206976 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.193s user 0.069s sys 0.045s Metrics: {"bytes_written":34609,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25578,"cfile_cache_miss":31,"cfile_cache_miss_bytes":38666,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2217,"dirs.run_cpu_time_us":1338,"dirs.run_wall_time_us":5976,"drs_written":1,"lbm_read_time_us":4198,"lbm_reads_lt_1ms":63,"lbm_write_time_us":4883,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":152,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":133,"thread_start_us":57045,"threads_started":2}
I20260807 07:53:17.208635 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000061
I20260807 07:53:17.225322 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.004s sys 0.008s Metrics: {"bytes_written":5192,"cfile_init":1,"dirs.queue_time_us":337,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":1277,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":928,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.226234 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6366 bytes on disk
I20260807 07:53:17.227200 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":177,"lbm_reads_lt_1ms":8}
I20260807 07:53:17.228001 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.175950
I20260807 07:53:17.262069 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.020s sys 0.013s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25953,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38873,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":833,"dirs.run_cpu_time_us":759,"dirs.run_wall_time_us":3865,"drs_written":1,"lbm_read_time_us":989,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1697,"lbm_writes_lt_1ms":32,"mutex_wait_us":63,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":135,"spinlock_wait_cycles":14592,"thread_start_us":1018,"threads_started":1,"update_count":1}
I20260807 07:53:17.263602 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6091 bytes on disk
I20260807 07:53:17.267802 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.279152 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33900:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.294600 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 3b7c7026f90947a6a96bf357bed5f589 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.67 [id=71451f1244d1417bb54a2bda7e7e9c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.295220 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7c7026f90947a6a96bf357bed5f589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.303082 10215 tablet_bootstrap.cc:492] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.305121 10215 tablet_bootstrap.cc:654] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.308696 10215 tablet_bootstrap.cc:492] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.308938 10215 ts_tablet_manager.cc:1403] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:17.309828 10215 raft_consensus.cc:359] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.310007 10215 raft_consensus.cc:385] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.310065 10215 raft_consensus.cc:740] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.310447 10215 consensus_queue.cc:260] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.310640 10215 raft_consensus.cc:399] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.310768 10215 raft_consensus.cc:493] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.310897 10215 raft_consensus.cc:3060] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.313254 10215 raft_consensus.cc:515] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.313582 10215 leader_election.cc:304] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.313899 10215 leader_election.cc:290] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.313939 11009 raft_consensus.cc:2804] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.314116 11009 raft_consensus.cc:697] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.314528 11009 consensus_queue.cc:237] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.315223 10215 ts_tablet_manager.cc:1434] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 07:53:17.322754 9887 catalog_manager.cc:5697] T 3b7c7026f90947a6a96bf357bed5f589 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.384995 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33906:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.399541 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab30f11261b466ea1e0983b45d73bb1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.68 [id=b7a8e93d55a748189c9815b2a6c9c668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.400063 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab30f11261b466ea1e0983b45d73bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.406873 10215 tablet_bootstrap.cc:492] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.409883 10215 tablet_bootstrap.cc:654] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.413694 10215 tablet_bootstrap.cc:492] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.413918 10215 ts_tablet_manager.cc:1403] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 07:53:17.414718 10215 raft_consensus.cc:359] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.414928 10215 raft_consensus.cc:385] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.415063 10215 raft_consensus.cc:740] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.415418 10215 consensus_queue.cc:260] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.415644 10215 raft_consensus.cc:399] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.415756 10215 raft_consensus.cc:493] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.415951 10215 raft_consensus.cc:3060] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.418757 10215 raft_consensus.cc:515] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.419104 10215 leader_election.cc:304] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.419368 10215 leader_election.cc:290] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.419526 11009 raft_consensus.cc:2804] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.419932 11009 raft_consensus.cc:697] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.420204 10215 ts_tablet_manager.cc:1434] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:17.420300 11009 consensus_queue.cc:237] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.430469 9887 catalog_manager.cc:5697] T 7ab30f11261b466ea1e0983b45d73bb1 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.446882 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000090
I20260807 07:53:17.454607 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:17.463505 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.009s sys 0.004s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":1300,"drs_written":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4,"lbm_write_time_us":821,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:17.464745 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 466 bytes on disk
I20260807 07:53:17.465682 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.466516 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.215783
I20260807 07:53:17.471385 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.012s sys 0.002s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":83,"dirs.run_cpu_time_us":237,"dirs.run_wall_time_us":1215,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1245,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:17.472434 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.215783
I20260807 07:53:17.515219 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33914:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.519300 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:17.532743 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.032s sys 0.009s Metrics: {"bytes_written":35914,"cfile_cache_hit":12,"cfile_cache_hit_bytes":26646,"cfile_cache_miss":28,"cfile_cache_miss_bytes":39910,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":8795,"dirs.run_cpu_time_us":1032,"dirs.run_wall_time_us":7501,"drs_written":1,"lbm_read_time_us":1639,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1993,"lbm_writes_lt_1ms":28,"mutex_wait_us":2065,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":139}
I20260807 07:53:17.533905 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6286 bytes on disk
I20260807 07:53:17.533931 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 7146e59ba6e74a1b94b93eb8ac3b319e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.69 [id=5938c4c9783b4e41907fea4dcf77f662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.534518 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7146e59ba6e74a1b94b93eb8ac3b319e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.537575 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":2493,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260807 07:53:17.538421 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000021
I20260807 07:53:17.542701 10215 tablet_bootstrap.cc:492] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.543306 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.007s sys 0.009s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":142,"dirs.run_wall_time_us":1035,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1554,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4,"spinlock_wait_cycles":11520}
I20260807 07:53:17.544294 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.215334
I20260807 07:53:17.544803 10215 tablet_bootstrap.cc:654] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.553632 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.008s sys 0.007s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":126,"dirs.run_wall_time_us":949,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1188,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.554679 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.200980
I20260807 07:53:17.556481 10215 tablet_bootstrap.cc:492] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.556813 10215 ts_tablet_manager.cc:1403] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260807 07:53:17.559333 10215 raft_consensus.cc:359] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.559635 10215 raft_consensus.cc:385] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.559780 10215 raft_consensus.cc:740] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.560233 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.088s user 0.036s sys 0.015s Metrics: {"bytes_written":35914,"cfile_cache_hit":12,"cfile_cache_hit_bytes":26646,"cfile_cache_miss":28,"cfile_cache_miss_bytes":39910,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":592,"dirs.run_cpu_time_us":822,"dirs.run_wall_time_us":5039,"drs_written":1,"lbm_read_time_us":10172,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":59,"lbm_write_time_us":1778,"lbm_writes_lt_1ms":28,"mutex_wait_us":169,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":139}
I20260807 07:53:17.560606 10215 consensus_queue.cc:260] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.560931 10215 raft_consensus.cc:399] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.561095 10215 raft_consensus.cc:493] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.561182 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000025
I20260807 07:53:17.561286 10215 raft_consensus.cc:3060] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.564571 10215 raft_consensus.cc:515] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.566075 10215 leader_election.cc:304] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.566380 11009 raft_consensus.cc:2804] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.566563 11009 raft_consensus.cc:697] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.566843 11009 consensus_queue.cc:237] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.572796 10215 leader_election.cc:290] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.573243 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.002s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":126,"dirs.run_wall_time_us":808,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1037,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.574216 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.200980
I20260807 07:53:17.574966 10215 ts_tablet_manager.cc:1434] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.018s user 0.004s sys 0.003s
I20260807 07:53:17.598483 9886 catalog_manager.cc:5697] T 7146e59ba6e74a1b94b93eb8ac3b319e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.598847 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.029s sys 0.004s Metrics: {"bytes_written":36319,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27023,"cfile_cache_miss":28,"cfile_cache_miss_bytes":40263,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":4091,"dirs.run_cpu_time_us":803,"dirs.run_wall_time_us":4404,"drs_written":1,"lbm_read_time_us":1329,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1227,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":141}
I20260807 07:53:17.600813 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6360 bytes on disk
I20260807 07:53:17.601545 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.603458 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.032s sys 0.019s Metrics: {"bytes_written":35914,"cfile_cache_hit":13,"cfile_cache_hit_bytes":26714,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40420,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2547,"dirs.run_cpu_time_us":1029,"dirs.run_wall_time_us":10264,"drs_written":1,"lbm_read_time_us":1634,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1697,"lbm_writes_lt_1ms":28,"mutex_wait_us":1377,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":139,"thread_start_us":17216,"threads_started":1}
I20260807 07:53:17.607280 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000018
I20260807 07:53:17.617175 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.029s sys 0.003s Metrics: {"bytes_written":36319,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27023,"cfile_cache_miss":28,"cfile_cache_miss_bytes":40263,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":1898,"dirs.run_cpu_time_us":707,"dirs.run_wall_time_us":7048,"drs_written":1,"lbm_read_time_us":1390,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1102,"lbm_writes_lt_1ms":28,"mutex_wait_us":43,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":141,"spinlock_wait_cycles":4352,"thread_start_us":1627,"threads_started":2}
I20260807 07:53:17.618031 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:17.625326 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.004s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":186,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:17.625946 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260807 07:53:17.629562 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.011s sys 0.004s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":143,"dirs.run_cpu_time_us":233,"dirs.run_wall_time_us":1155,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":971,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:17.630767 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.003s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":101,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:17.631234 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:17.633169 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6521 bytes on disk
I20260807 07:53:17.634130 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":154,"lbm_reads_lt_1ms":8}
I20260807 07:53:17.634985 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:17.638072 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.004s sys 0.003s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":74,"lbm_read_time_us":191,"lbm_reads_lt_1ms":12,"lbm_write_time_us":166,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:17.638724 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6360 bytes on disk
I20260807 07:53:17.639261 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.663160 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.027s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":77,"lbm_read_time_us":331,"lbm_reads_lt_1ms":17,"lbm_write_time_us":407,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:17.664119 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.208807
I20260807 07:53:17.678498 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33918:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.692453 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a6890a310c06496899e1086cdb8b527b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.70 [id=9ef8f8bbe128437d9849f15eb809c7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.692945 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6890a310c06496899e1086cdb8b527b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.697760 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.030s sys 0.000s Metrics: {"bytes_written":36356,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27091,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40721,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1911,"dirs.run_cpu_time_us":763,"dirs.run_wall_time_us":6461,"drs_written":1,"lbm_read_time_us":1548,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1103,"lbm_writes_lt_1ms":28,"mutex_wait_us":70,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":141,"spinlock_wait_cycles":11904,"thread_start_us":815,"threads_started":1}
I20260807 07:53:17.698668 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000019
I20260807 07:53:17.699321 10215 tablet_bootstrap.cc:492] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.701635 10215 tablet_bootstrap.cc:654] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.704890 10215 tablet_bootstrap.cc:492] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.705039 10215 ts_tablet_manager.cc:1403] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260807 07:53:17.705801 10215 raft_consensus.cc:359] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.705993 10215 raft_consensus.cc:385] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.706101 10215 raft_consensus.cc:740] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.706380 10215 consensus_queue.cc:260] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.706552 10215 raft_consensus.cc:399] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.706647 10215 raft_consensus.cc:493] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.706790 10215 raft_consensus.cc:3060] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.709156 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":78,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":999,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":630,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:17.709672 10215 raft_consensus.cc:515] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.710012 10215 leader_election.cc:304] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.710126 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6623 bytes on disk
I20260807 07:53:17.710297 10215 leader_election.cc:290] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.710386 11009 raft_consensus.cc:2804] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.710678 11009 raft_consensus.cc:697] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.711018 10215 ts_tablet_manager.cc:1434] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:53:17.711016 11009 consensus_queue.cc:237] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.711585 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":137,"lbm_reads_lt_1ms":8}
I20260807 07:53:17.712404 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223842
I20260807 07:53:17.727205 9886 catalog_manager.cc:5697] T a6890a310c06496899e1086cdb8b527b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.745601 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.027s sys 0.004s Metrics: {"bytes_written":36749,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27398,"cfile_cache_miss":28,"cfile_cache_miss_bytes":40844,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":323,"dirs.run_cpu_time_us":798,"dirs.run_wall_time_us":4111,"drs_written":1,"lbm_read_time_us":1235,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1784,"lbm_writes_lt_1ms":28,"mutex_wait_us":2,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":143,"spinlock_wait_cycles":1280}
I20260807 07:53:17.746373 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6462 bytes on disk
I20260807 07:53:17.746973 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.747553 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:17.751561 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":125,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:17.775290 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33928:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.789252 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 2aae2af2c43a4341a8b3f3521f3bd73a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.71 [id=54d16071efaf454eaa1240d4cb23df24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.789821 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aae2af2c43a4341a8b3f3521f3bd73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.795481 10215 tablet_bootstrap.cc:492] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.798178 10215 tablet_bootstrap.cc:654] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.801606 10215 tablet_bootstrap.cc:492] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.801859 10215 ts_tablet_manager.cc:1403] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 07:53:17.802687 10215 raft_consensus.cc:359] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.802893 10215 raft_consensus.cc:385] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.802996 10215 raft_consensus.cc:740] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.803347 10215 consensus_queue.cc:260] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.803553 10215 raft_consensus.cc:399] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.803683 10215 raft_consensus.cc:493] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.803798 10215 raft_consensus.cc:3060] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.806109 10215 raft_consensus.cc:515] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.806370 10215 leader_election.cc:304] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.806675 10215 leader_election.cc:290] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.806722 11009 raft_consensus.cc:2804] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.806903 11009 raft_consensus.cc:697] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.807289 11009 consensus_queue.cc:237] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.807638 10215 ts_tablet_manager.cc:1434] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:17.814499 9886 catalog_manager.cc:5697] T 2aae2af2c43a4341a8b3f3521f3bd73a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.854490 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:17.855798 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33940:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.870345 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.012s sys 0.001s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1194,"drs_written":1,"lbm_read_time_us":705,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1446,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:17.871764 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 437 bytes on disk
I20260807 07:53:17.871968 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 295f946c1c2f4347bdb9fc7b152c446a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.72 [id=b4dac50e13b24000bb4d5918a348822b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.872556 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295f946c1c2f4347bdb9fc7b152c446a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:17.872920 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":499,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.874071 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.239935
I20260807 07:53:17.878800 10215 tablet_bootstrap.cc:492] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:17.880956 10215 tablet_bootstrap.cc:654] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:17.884665 10215 tablet_bootstrap.cc:492] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:17.884838 10215 ts_tablet_manager.cc:1403] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 07:53:17.886059 10215 raft_consensus.cc:359] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.886267 10215 raft_consensus.cc:385] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:17.886432 10215 raft_consensus.cc:740] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:17.886703 10215 consensus_queue.cc:260] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.886938 10215 raft_consensus.cc:399] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:17.887065 10215 raft_consensus.cc:493] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:17.887231 10215 raft_consensus.cc:3060] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:17.889847 10215 raft_consensus.cc:515] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.890130 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260807 07:53:17.890187 10215 leader_election.cc:304] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:17.890427 10215 leader_election.cc:290] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:17.890492 11009 raft_consensus.cc:2804] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:17.890833 11009 raft_consensus.cc:697] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:17.891204 10215 ts_tablet_manager.cc:1434] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 07:53:17.891271 11009 consensus_queue.cc:237] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:17.901782 9886 catalog_manager.cc:5697] T 295f946c1c2f4347bdb9fc7b152c446a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:17.911720 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.011s sys 0.001s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1198,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1905,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:17.912741 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248285
I20260807 07:53:17.933565 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.059s user 0.030s sys 0.003s Metrics: {"bytes_written":37234,"cfile_cache_hit":12,"cfile_cache_hit_bytes":27761,"cfile_cache_miss":28,"cfile_cache_miss_bytes":41551,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":3933,"dirs.run_cpu_time_us":671,"dirs.run_wall_time_us":5457,"drs_written":1,"lbm_read_time_us":1100,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1062,"lbm_writes_lt_1ms":28,"mutex_wait_us":77,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":145,"thread_start_us":5357,"threads_started":1}
I20260807 07:53:17.934582 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6554 bytes on disk
I20260807 07:53:17.935186 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.935730 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000018
I20260807 07:53:17.948318 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.004s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":131,"dirs.run_wall_time_us":1268,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1013,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":896}
I20260807 07:53:17.949354 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 359 bytes on disk
I20260807 07:53:17.949877 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.950639 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.173252
I20260807 07:53:17.962960 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.027s sys 0.010s Metrics: {"bytes_written":37656,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28131,"cfile_cache_miss":31,"cfile_cache_miss_bytes":42571,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2018,"dirs.run_cpu_time_us":906,"dirs.run_wall_time_us":5170,"drs_written":1,"lbm_read_time_us":1415,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1194,"lbm_writes_lt_1ms":28,"mutex_wait_us":368,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":147,"thread_start_us":6713,"threads_started":1}
I20260807 07:53:17.963968 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6632 bytes on disk
I20260807 07:53:17.964699 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.982654 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33954:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:17.990782 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.029s sys 0.008s Metrics: {"bytes_written":37656,"cfile_cache_hit":12,"cfile_cache_hit_bytes":28176,"cfile_cache_miss":28,"cfile_cache_miss_bytes":42200,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1718,"dirs.run_cpu_time_us":561,"dirs.run_wall_time_us":2584,"drs_written":1,"lbm_read_time_us":1456,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1651,"lbm_writes_lt_1ms":28,"mutex_wait_us":25,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":147,"spinlock_wait_cycles":896,"thread_start_us":602,"threads_started":1}
I20260807 07:53:17.992070 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6632 bytes on disk
I20260807 07:53:17.992662 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260807 07:53:17.993798 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:17.999081 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 803e15ab2d0241ad89a2bb0ebbdc16c4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.73 [id=8588a6dde7af4578a0a9ba76bc7b11af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:17.999663 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 803e15ab2d0241ad89a2bb0ebbdc16c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.002439 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:18.006886 10215 tablet_bootstrap.cc:492] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.009747 10215 tablet_bootstrap.cc:654] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.014163 10215 tablet_bootstrap.cc:492] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.014433 10215 ts_tablet_manager.cc:1403] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 07:53:18.016623 10215 raft_consensus.cc:359] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.016891 10215 raft_consensus.cc:385] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.017053 10215 raft_consensus.cc:740] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.017506 10215 consensus_queue.cc:260] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.017710 10215 raft_consensus.cc:399] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.017870 10215 raft_consensus.cc:493] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.018018 10215 raft_consensus.cc:3060] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.018429 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.009s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":149,"dirs.run_wall_time_us":942,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1262,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:18.019187 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 505 bytes on disk
I20260807 07:53:18.019397 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.008s sys 0.008s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":82,"dirs.run_cpu_time_us":259,"dirs.run_wall_time_us":1175,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1598,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:18.020232 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.020527 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:18.021615 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223287
I20260807 07:53:18.022207 10215 raft_consensus.cc:515] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.022548 10215 leader_election.cc:304] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.023084 11009 raft_consensus.cc:2804] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.023355 11009 raft_consensus.cc:697] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.023867 11009 consensus_queue.cc:237] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.032343 9886 catalog_manager.cc:5697] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.034118 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"lbm_read_time_us":600,"lbm_reads_lt_1ms":25,"lbm_write_time_us":577,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:18.034711 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.201912
I20260807 07:53:18.037472 10215 leader_election.cc:290] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.044780 10215 ts_tablet_manager.cc:1434] T 803e15ab2d0241ad89a2bb0ebbdc16c4 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.030s user 0.001s sys 0.007s
I20260807 07:53:18.066390 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.031s sys 0.008s Metrics: {"bytes_written":38066,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28496,"cfile_cache_miss":31,"cfile_cache_miss_bytes":43030,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1144,"dirs.run_cpu_time_us":906,"dirs.run_wall_time_us":6105,"drs_written":1,"lbm_read_time_us":1630,"lbm_reads_lt_1ms":63,"lbm_write_time_us":3489,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":16,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":149}
I20260807 07:53:18.067428 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6701 bytes on disk
I20260807 07:53:18.068032 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.068594 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000004
I20260807 07:53:18.076804 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.002s Metrics: {"bytes_written":391,"delete_count":0,"lbm_write_time_us":307,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:18.082954 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.027s sys 0.012s Metrics: {"bytes_written":391,"cfile_cache_hit":13,"cfile_cache_hit_bytes":28536,"cfile_cache_miss":31,"cfile_cache_miss_bytes":42500,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":2143,"dirs.run_cpu_time_us":1343,"dirs.run_wall_time_us":5665,"drs_written":1,"lbm_read_time_us":1614,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1828,"lbm_writes_lt_1ms":32,"mutex_wait_us":667,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":149,"spinlock_wait_cycles":8064,"update_count":2}
I20260807 07:53:18.083681 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6701 bytes on disk
I20260807 07:53:18.084196 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.106175 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33958:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.121170 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 8097fca180a3405bbce37b7594c54020 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.74 [id=a89687569bf443f6a1f3f772b49f52c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.121763 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8097fca180a3405bbce37b7594c54020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.128939 10215 tablet_bootstrap.cc:492] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.132211 10215 tablet_bootstrap.cc:654] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.136488 10215 tablet_bootstrap.cc:492] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.136690 10215 ts_tablet_manager.cc:1403] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 07:53:18.137917 10215 raft_consensus.cc:359] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.138165 10215 raft_consensus.cc:385] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.138350 10215 raft_consensus.cc:740] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.138669 10215 consensus_queue.cc:260] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.138928 10215 raft_consensus.cc:399] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.139057 10215 raft_consensus.cc:493] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.139232 10215 raft_consensus.cc:3060] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.142522 10215 raft_consensus.cc:515] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.142905 10215 leader_election.cc:304] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.143239 10215 leader_election.cc:290] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.143342 11009 raft_consensus.cc:2804] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.143771 11009 raft_consensus.cc:697] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.143898 10215 ts_tablet_manager.cc:1434] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 07:53:18.144236 11009 consensus_queue.cc:237] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.152281 9886 catalog_manager.cc:5697] T 8097fca180a3405bbce37b7594c54020 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.204493 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33972:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.215694 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:18.217891 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 267ad336b7d64db59447732313f7fe90 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.75 [id=7c331cc513ff4ae59a7fe34ea89f7078]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.218326 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267ad336b7d64db59447732313f7fe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.224740 10215 tablet_bootstrap.cc:492] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.227463 10215 tablet_bootstrap.cc:654] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.227455 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":6030,"cfile_init":1,"dirs.queue_time_us":99,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1408,"drs_written":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4,"lbm_write_time_us":657,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:18.228240 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 532 bytes on disk
I20260807 07:53:18.228958 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.229607 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.221823
I20260807 07:53:18.230876 10215 tablet_bootstrap.cc:492] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.231091 10215 ts_tablet_manager.cc:1403] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:18.232019 10215 raft_consensus.cc:359] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.232357 10215 raft_consensus.cc:385] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.232599 10215 raft_consensus.cc:740] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.232970 10215 consensus_queue.cc:260] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.233114 10215 raft_consensus.cc:399] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.233201 10215 raft_consensus.cc:493] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.233376 10215 raft_consensus.cc:3060] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.236177 10215 raft_consensus.cc:515] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.236485 10215 leader_election.cc:304] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.236937 11009 raft_consensus.cc:2804] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.237212 11009 raft_consensus.cc:697] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.237636 11009 consensus_queue.cc:237] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.250465 10215 leader_election.cc:290] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.251174 10215 ts_tablet_manager.cc:1434] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260807 07:53:18.252544 9887 catalog_manager.cc:5697] T 267ad336b7d64db59447732313f7fe90 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.272706 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.023s sys 0.013s Metrics: {"bytes_written":38942,"cfile_cache_hit":12,"cfile_cache_hit_bytes":29239,"cfile_cache_miss":28,"cfile_cache_miss_bytes":43757,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":851,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":1301,"drs_written":1,"lbm_read_time_us":1753,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1532,"lbm_writes_lt_1ms":28,"mutex_wait_us":49,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":153}
I20260807 07:53:18.273707 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260807 07:53:18.283957 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.001s sys 0.004s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":190,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:18.284757 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6874 bytes on disk
I20260807 07:53:18.286289 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.287181 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:18.291563 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.001s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":217,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:18.292110 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:18.298611 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.001s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":58,"lbm_read_time_us":139,"lbm_reads_lt_1ms":12,"lbm_write_time_us":153,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:18.330889 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:18.335186 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260807 07:53:18.344801 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33982:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.347935 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.014s sys 0.000s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":253,"dirs.run_wall_time_us":1393,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1471,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:18.348721 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 465 bytes on disk
I20260807 07:53:18.350183 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.350893 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.212324
I20260807 07:53:18.353152 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.012s sys 0.001s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":255,"dirs.run_wall_time_us":1188,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1324,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:18.354272 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 465 bytes on disk
I20260807 07:53:18.354722 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.355247 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.212324
I20260807 07:53:18.364018 10099 tablet_service.cc:1511] Processing CreateTablet for tablet db306113e93d4c3889209bb499f59c43 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.76 [id=8883259252f7481c96f688f731b6a483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.364602 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db306113e93d4c3889209bb499f59c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.374531 10215 tablet_bootstrap.cc:492] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.377230 10215 tablet_bootstrap.cc:654] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.381744 10215 tablet_bootstrap.cc:492] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.381978 10215 ts_tablet_manager.cc:1403] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260807 07:53:18.383288 10215 raft_consensus.cc:359] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.383598 10215 raft_consensus.cc:385] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.383803 10215 raft_consensus.cc:740] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.384204 10215 consensus_queue.cc:260] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.384586 10215 raft_consensus.cc:399] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.384770 10215 raft_consensus.cc:493] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.385118 10215 raft_consensus.cc:3060] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.386943 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.034s sys 0.000s Metrics: {"bytes_written":38979,"cfile_cache_hit":13,"cfile_cache_hit_bytes":29275,"cfile_cache_miss":31,"cfile_cache_miss_bytes":44315,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1210,"dirs.run_cpu_time_us":309,"dirs.run_wall_time_us":3861,"drs_written":1,"lbm_read_time_us":2050,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1373,"lbm_writes_lt_1ms":28,"mutex_wait_us":37,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":153,"spinlock_wait_cycles":27904,"thread_start_us":303,"threads_started":1}
I20260807 07:53:18.387884 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6913 bytes on disk
I20260807 07:53:18.388562 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.389238 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:18.390945 10215 raft_consensus.cc:515] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.391431 10215 leader_election.cc:304] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.391710 10215 leader_election.cc:290] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.391844 11009 raft_consensus.cc:2804] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.392154 11009 raft_consensus.cc:697] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.392561 11009 consensus_queue.cc:237] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.394428 10215 ts_tablet_manager.cc:1434] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.012s user 0.001s sys 0.007s
I20260807 07:53:18.402022 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.025s sys 0.015s Metrics: {"bytes_written":38979,"cfile_cache_hit":13,"cfile_cache_hit_bytes":29275,"cfile_cache_miss":31,"cfile_cache_miss_bytes":44315,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":2766,"dirs.run_cpu_time_us":482,"dirs.run_wall_time_us":2613,"drs_written":1,"lbm_read_time_us":1566,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1979,"lbm_writes_lt_1ms":28,"mutex_wait_us":2446,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":153,"spinlock_wait_cycles":1280}
I20260807 07:53:18.403235 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000014
I20260807 07:53:18.405325 9887 catalog_manager.cc:5697] T db306113e93d4c3889209bb499f59c43 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.414492 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.016s sys 0.007s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1096,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1411,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:18.415371 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:18.416209 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.416893 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224859
I20260807 07:53:18.434351 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.019s sys 0.003s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":1143,"drs_written":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2671,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:18.435225 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7148 bytes on disk
I20260807 07:53:18.435855 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":125,"lbm_reads_lt_1ms":8}
I20260807 07:53:18.436440 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.459198
I20260807 07:53:18.442843 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.003s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":576,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":181,"lbm_read_time_us":212,"lbm_reads_lt_1ms":17,"lbm_write_time_us":204,"lbm_writes_lt_1ms":9,"mutex_wait_us":22,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":2944,"update_count":1}
I20260807 07:53:18.443395 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231807
I20260807 07:53:18.444525 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.003s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":29618,"cfile_cache_miss":28,"cfile_cache_miss_bytes":44122,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":989,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":1206,"drs_written":1,"lbm_read_time_us":973,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1044,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":155,"spinlock_wait_cycles":1792,"update_count":1}
I20260807 07:53:18.445581 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7000 bytes on disk
I20260807 07:53:18.446146 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.474566 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.027s sys 0.001s Metrics: {"bytes_written":39403,"cfile_cache_hit":13,"cfile_cache_hit_bytes":29686,"cfile_cache_miss":31,"cfile_cache_miss_bytes":44580,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":955,"dirs.run_cpu_time_us":956,"dirs.run_wall_time_us":5991,"drs_written":1,"lbm_read_time_us":1151,"lbm_reads_lt_1ms":63,"lbm_write_time_us":884,"lbm_writes_lt_1ms":28,"mutex_wait_us":81,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":155,"spinlock_wait_cycles":4480,"thread_start_us":5083,"threads_started":1}
I20260807 07:53:18.475865 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7011 bytes on disk
I20260807 07:53:18.477480 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":420,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.482311 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33990:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.495508 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f89063fa8453419cb1a8d6ef60e250f7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.77 [id=81d56d3e4c424e1ebea8049702d8d3f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.495954 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89063fa8453419cb1a8d6ef60e250f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.500378 10215 tablet_bootstrap.cc:492] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.502051 10215 tablet_bootstrap.cc:654] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.504530 10215 tablet_bootstrap.cc:492] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.504724 10215 ts_tablet_manager.cc:1403] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 07:53:18.505509 10215 raft_consensus.cc:359] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.505692 10215 raft_consensus.cc:385] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.505812 10215 raft_consensus.cc:740] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.506105 10215 consensus_queue.cc:260] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.506253 10215 raft_consensus.cc:399] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.506377 10215 raft_consensus.cc:493] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.506490 10215 raft_consensus.cc:3060] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.508736 10215 raft_consensus.cc:515] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.508970 10215 leader_election.cc:304] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.509219 10215 leader_election.cc:290] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.509385 10219 raft_consensus.cc:2804] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.509734 10215 ts_tablet_manager.cc:1434] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 07:53:18.510241 10219 raft_consensus.cc:697] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.510517 10219 consensus_queue.cc:237] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.516558 9886 catalog_manager.cc:5697] T f89063fa8453419cb1a8d6ef60e250f7 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.549989 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000090
I20260807 07:53:18.556389 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33992:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.571557 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 10a04af2d48a42a3a0e70f03cdabf6e6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.78 [id=de751dea5e9844ee89d96e50f5fe1e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.572167 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a04af2d48a42a3a0e70f03cdabf6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.579802 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.009s sys 0.015s Metrics: {"bytes_written":5627,"cfile_init":1,"dirs.queue_time_us":542,"dirs.run_cpu_time_us":207,"dirs.run_wall_time_us":2527,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4146,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:18.580084 10215 tablet_bootstrap.cc:492] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.580762 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238789
I20260807 07:53:18.581763 10215 tablet_bootstrap.cc:654] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.584800 10215 tablet_bootstrap.cc:492] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.584971 10215 ts_tablet_manager.cc:1403] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:18.586082 10215 raft_consensus.cc:359] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.586282 10215 raft_consensus.cc:385] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.586378 10215 raft_consensus.cc:740] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.586637 10215 consensus_queue.cc:260] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.586853 10215 raft_consensus.cc:399] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.586947 10215 raft_consensus.cc:493] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.587075 10215 raft_consensus.cc:3060] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.589159 10215 raft_consensus.cc:515] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.589438 10215 leader_election.cc:304] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.589633 10215 leader_election.cc:290] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.589726 10219 raft_consensus.cc:2804] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.589985 10219 raft_consensus.cc:697] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.590217 10219 consensus_queue.cc:237] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.590487 10215 ts_tablet_manager.cc:1434] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 07:53:18.599712 9886 catalog_manager.cc:5697] T 10a04af2d48a42a3a0e70f03cdabf6e6 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.610204 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.028s sys 0.000s Metrics: {"bytes_written":39848,"cfile_cache_hit":13,"cfile_cache_hit_bytes":30015,"cfile_cache_miss":31,"cfile_cache_miss_bytes":45407,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":768,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1018,"drs_written":1,"lbm_read_time_us":1345,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1102,"lbm_writes_lt_1ms":28,"mutex_wait_us":73,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":157,"spinlock_wait_cycles":62592}
I20260807 07:53:18.610993 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260807 07:53:18.620713 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.000s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":40,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1180,"drs_written":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4,"lbm_write_time_us":709,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:18.621474 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.169327
I20260807 07:53:18.654484 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.027s sys 0.004s Metrics: {"bytes_written":40280,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30396,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45502,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":436,"dirs.run_cpu_time_us":950,"dirs.run_wall_time_us":5174,"drs_written":1,"lbm_read_time_us":1241,"lbm_reads_lt_1ms":64,"lbm_write_time_us":948,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":159}
I20260807 07:53:18.655434 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7182 bytes on disk
I20260807 07:53:18.656070 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.668324 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33998:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.683269 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 780df1ac92f144e3b8ffe13b42a28e1b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.79 [id=1c51d4cce2ce467eb968601b43361e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.683755 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 780df1ac92f144e3b8ffe13b42a28e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.690092 10215 tablet_bootstrap.cc:492] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.692924 10215 tablet_bootstrap.cc:654] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.696452 10215 tablet_bootstrap.cc:492] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.696683 10215 ts_tablet_manager.cc:1403] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 07:53:18.697206 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260807 07:53:18.697994 10215 raft_consensus.cc:359] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.698233 10215 raft_consensus.cc:385] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.698348 10215 raft_consensus.cc:740] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.698762 10215 consensus_queue.cc:260] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.699003 10215 raft_consensus.cc:399] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.699222 10215 raft_consensus.cc:493] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.699394 10215 raft_consensus.cc:3060] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.702931 10215 raft_consensus.cc:515] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.703231 10215 leader_election.cc:304] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.703630 10215 leader_election.cc:290] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.703733 10219 raft_consensus.cc:2804] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.704015 10219 raft_consensus.cc:697] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.704341 10215 ts_tablet_manager.cc:1434] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 07:53:18.704525 10219 consensus_queue.cc:237] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.712416 9886 catalog_manager.cc:5697] T 780df1ac92f144e3b8ffe13b42a28e1b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.717572 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.008s sys 0.005s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":464,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1256,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1013,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:18.718509 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249982
I20260807 07:53:18.728483 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:18.740330 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1336,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":892,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:18.741144 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.251280
I20260807 07:53:18.750671 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.027s sys 0.000s Metrics: {"bytes_written":40733,"cfile_cache_hit":13,"cfile_cache_hit_bytes":30750,"cfile_cache_miss":31,"cfile_cache_miss_bytes":46370,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":716,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":1155,"drs_written":1,"lbm_read_time_us":1287,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1298,"lbm_writes_lt_1ms":28,"mutex_wait_us":2,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":161,"spinlock_wait_cycles":105856}
I20260807 07:53:18.751564 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7269 bytes on disk
I20260807 07:53:18.752229 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.773244 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.026s sys 0.003s Metrics: {"bytes_written":40733,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30756,"cfile_cache_miss":28,"cfile_cache_miss_bytes":46018,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":876,"dirs.run_cpu_time_us":265,"dirs.run_wall_time_us":1201,"drs_written":1,"lbm_read_time_us":1295,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1237,"lbm_writes_lt_1ms":28,"mutex_wait_us":2,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":161}
I20260807 07:53:18.774413 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7269 bytes on disk
I20260807 07:53:18.774307 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34000:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.775115 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.789089 10099 tablet_service.cc:1511] Processing CreateTablet for tablet aa74de43c54a4197a0bec5ff3c3a43ac (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.80 [id=42d9a80b98e041adbec9c782473d82ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.789654 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa74de43c54a4197a0bec5ff3c3a43ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.795974 10215 tablet_bootstrap.cc:492] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.798199 10215 tablet_bootstrap.cc:654] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.801162 10215 tablet_bootstrap.cc:492] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.801405 10215 ts_tablet_manager.cc:1403] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 07:53:18.802557 10215 raft_consensus.cc:359] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.802783 10215 raft_consensus.cc:385] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.803007 10215 raft_consensus.cc:740] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.803323 10215 consensus_queue.cc:260] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.803546 10215 raft_consensus.cc:399] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.803663 10215 raft_consensus.cc:493] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.803830 10215 raft_consensus.cc:3060] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.806074 10215 raft_consensus.cc:515] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.806355 10215 leader_election.cc:304] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.806578 10215 leader_election.cc:290] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.806702 10219 raft_consensus.cc:2804] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.807469 10215 ts_tablet_manager.cc:1434] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 07:53:18.807600 10219 raft_consensus.cc:697] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.807948 10219 consensus_queue.cc:237] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.814641 9886 catalog_manager.cc:5697] T aa74de43c54a4197a0bec5ff3c3a43ac P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.868186 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34004:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.881721 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f652db07f1fb4f3aa54db5ac188f05a2 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.81 [id=20cc4fb458cf4730a1806cd469ac20e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.882251 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f652db07f1fb4f3aa54db5ac188f05a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.888268 10215 tablet_bootstrap.cc:492] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.891044 10215 tablet_bootstrap.cc:654] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.894819 10215 tablet_bootstrap.cc:492] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.895041 10215 ts_tablet_manager.cc:1403] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 07:53:18.895849 10215 raft_consensus.cc:359] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.896019 10215 raft_consensus.cc:385] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:18.896113 10215 raft_consensus.cc:740] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:18.896363 10215 consensus_queue.cc:260] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.896519 10215 raft_consensus.cc:399] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:18.896652 10215 raft_consensus.cc:493] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:18.896785 10215 raft_consensus.cc:3060] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:18.899430 10215 raft_consensus.cc:515] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.899694 10215 leader_election.cc:304] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:18.900115 10215 leader_election.cc:290] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:18.900125 10219 raft_consensus.cc:2804] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:18.900369 10219 raft_consensus.cc:697] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:18.900693 10215 ts_tablet_manager.cc:1434] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 07:53:18.900669 10219 consensus_queue.cc:237] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:18.907066 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:18.908275 9886 catalog_manager.cc:5697] T f652db07f1fb4f3aa54db5ac188f05a2 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:18.923065 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.014s sys 0.000s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":203,"dirs.run_wall_time_us":1141,"drs_written":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1439,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:18.923954 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.255750
I20260807 07:53:18.959038 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.022s sys 0.011s Metrics: {"bytes_written":41606,"cfile_cache_hit":13,"cfile_cache_hit_bytes":31493,"cfile_cache_miss":31,"cfile_cache_miss_bytes":47473,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2000,"dirs.run_cpu_time_us":410,"dirs.run_wall_time_us":1817,"drs_written":1,"lbm_read_time_us":1244,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1180,"lbm_writes_lt_1ms":28,"mutex_wait_us":1246,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":165}
I20260807 07:53:18.960253 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7431 bytes on disk
I20260807 07:53:18.961491 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":138,"lbm_reads_lt_1ms":4}
I20260807 07:53:18.971169 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34014:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:18.985616 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 1089731288a24a7e951411684e662ba8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.82 [id=22f7bc8e2c48476ba3791b802d76bf4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:18.986125 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1089731288a24a7e951411684e662ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:18.993165 10215 tablet_bootstrap.cc:492] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:18.995381 10215 tablet_bootstrap.cc:654] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:18.999330 10215 tablet_bootstrap.cc:492] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:18.999542 10215 ts_tablet_manager.cc:1403] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 07:53:19.000931 10215 raft_consensus.cc:359] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.001255 10215 raft_consensus.cc:385] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.001562 10215 raft_consensus.cc:740] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.001930 10215 consensus_queue.cc:260] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.002323 10215 raft_consensus.cc:399] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.002456 10215 raft_consensus.cc:493] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.002704 10215 raft_consensus.cc:3060] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.003185 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:19.007354 10215 raft_consensus.cc:515] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.007884 10215 leader_election.cc:304] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.008189 10215 leader_election.cc:290] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.008311 10219 raft_consensus.cc:2804] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.008580 10219 raft_consensus.cc:697] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.008931 10215 ts_tablet_manager.cc:1434] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20260807 07:53:19.008872 10219 consensus_queue.cc:237] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.018687 9886 catalog_manager.cc:5697] T 1089731288a24a7e951411684e662ba8 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.023772 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.015s sys 0.000s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":1241,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1659,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:19.024621 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248373
I20260807 07:53:19.026050 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:19.037463 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.011s sys 0.000s Metrics: {"bytes_written":6068,"cfile_init":1,"dirs.queue_time_us":80,"dirs.run_cpu_time_us":266,"dirs.run_wall_time_us":1224,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1070,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:19.038192 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 517 bytes on disk
I20260807 07:53:19.038697 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.039222 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249286
I20260807 07:53:19.057068 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.027s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":31834,"cfile_cache_miss":28,"cfile_cache_miss_bytes":47608,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":525,"dirs.run_cpu_time_us":200,"dirs.run_wall_time_us":1053,"drs_written":1,"lbm_read_time_us":1237,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1282,"lbm_writes_lt_1ms":32,"mutex_wait_us":24,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":167,"spinlock_wait_cycles":5376,"update_count":1}
I20260807 07:53:19.057979 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7499 bytes on disk
I20260807 07:53:19.058506 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.070132 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.022s sys 0.007s Metrics: {"bytes_written":42030,"cfile_cache_hit":12,"cfile_cache_hit_bytes":31871,"cfile_cache_miss":28,"cfile_cache_miss_bytes":47667,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":644,"dirs.run_cpu_time_us":128,"dirs.run_wall_time_us":1149,"drs_written":1,"lbm_read_time_us":989,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1098,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":167}
I20260807 07:53:19.071442 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7516 bytes on disk
I20260807 07:53:19.072686 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":529,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.084070 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34030:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.100744 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 75fd6eeba59649bfb9e0b8d3dcbc6ef6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.83 [id=cbae7fdc035c4e66af8bdf720d23a818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.101263 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75fd6eeba59649bfb9e0b8d3dcbc6ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.107445 10215 tablet_bootstrap.cc:492] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.109881 10215 tablet_bootstrap.cc:654] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.113005 10215 tablet_bootstrap.cc:492] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.113231 10215 ts_tablet_manager.cc:1403] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:19.114059 10215 raft_consensus.cc:359] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.114244 10215 raft_consensus.cc:385] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.114315 10215 raft_consensus.cc:740] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.114589 10215 consensus_queue.cc:260] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.114723 10215 raft_consensus.cc:399] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.114809 10215 raft_consensus.cc:493] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.114923 10215 raft_consensus.cc:3060] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.117183 10215 raft_consensus.cc:515] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.117506 10215 leader_election.cc:304] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.117921 10215 leader_election.cc:290] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.118043 10219 raft_consensus.cc:2804] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.118328 10219 raft_consensus.cc:697] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.118664 10219 consensus_queue.cc:237] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.118920 10215 ts_tablet_manager.cc:1434] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 07:53:19.126920 9886 catalog_manager.cc:5697] T 75fd6eeba59649bfb9e0b8d3dcbc6ef6 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.173074 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34040:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.187196 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 4eed55059f5c4a1b939889512fcf9c4e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.84 [id=a6319ed1a8d24776a163cb45fdbe294d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.187747 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eed55059f5c4a1b939889512fcf9c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.193471 10215 tablet_bootstrap.cc:492] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.195585 10215 tablet_bootstrap.cc:654] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.198496 10215 tablet_bootstrap.cc:492] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.198684 10215 ts_tablet_manager.cc:1403] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:19.199642 10215 raft_consensus.cc:359] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.199888 10215 raft_consensus.cc:385] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.200090 10215 raft_consensus.cc:740] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.200402 10215 consensus_queue.cc:260] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.200629 10215 raft_consensus.cc:399] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.200747 10215 raft_consensus.cc:493] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.200909 10215 raft_consensus.cc:3060] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.203333 10215 raft_consensus.cc:515] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.203665 10215 leader_election.cc:304] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.203907 10215 leader_election.cc:290] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.204020 10219 raft_consensus.cc:2804] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.204234 10219 raft_consensus.cc:697] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.204512 10219 consensus_queue.cc:237] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.205958 10215 ts_tablet_manager.cc:1434] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 07:53:19.211910 9887 catalog_manager.cc:5697] T 4eed55059f5c4a1b939889512fcf9c4e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.212700 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:19.228819 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.005s Metrics: {"bytes_written":334,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":74,"dirs.run_cpu_time_us":198,"dirs.run_wall_time_us":1322,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1283,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:19.229717 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 511 bytes on disk
I20260807 07:53:19.230307 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.230983 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.228924
I20260807 07:53:19.241493 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1180,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":106,"lbm_read_time_us":416,"lbm_reads_lt_1ms":21,"lbm_write_time_us":382,"lbm_writes_lt_1ms":9,"mutex_wait_us":24,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":896,"update_count":1}
I20260807 07:53:19.242057 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.225806
I20260807 07:53:19.271056 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.027s sys 0.001s Metrics: {"bytes_written":42910,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32642,"cfile_cache_miss":31,"cfile_cache_miss_bytes":49136,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":545,"dirs.run_cpu_time_us":487,"dirs.run_wall_time_us":2751,"drs_written":1,"lbm_read_time_us":1127,"lbm_reads_lt_1ms":63,"lbm_write_time_us":981,"lbm_writes_lt_1ms":28,"mutex_wait_us":151,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":171,"spinlock_wait_cycles":2944}
I20260807 07:53:19.271989 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7697 bytes on disk
I20260807 07:53:19.272732 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.299541 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34044:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.309296 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:19.320394 10099 tablet_service.cc:1511] Processing CreateTablet for tablet a759131412044e27b064cc4401c14d4a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.85 [id=8a8b188aef3c41df8d91e7d28ab6b44b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.320950 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a759131412044e27b064cc4401c14d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.323809 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260807 07:53:19.328655 10215 tablet_bootstrap.cc:492] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.331344 10215 tablet_bootstrap.cc:654] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.331599 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.008s sys 0.005s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":198,"dirs.run_wall_time_us":6181,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":5087,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:19.332696 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.245353
I20260807 07:53:19.340922 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.015s sys 0.000s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1191,"drs_written":1,"lbm_read_time_us":1066,"lbm_reads_lt_1ms":4,"lbm_write_time_us":930,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:19.341910 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.240847
I20260807 07:53:19.343876 10215 tablet_bootstrap.cc:492] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.344170 10215 ts_tablet_manager.cc:1403] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260807 07:53:19.345441 10215 raft_consensus.cc:359] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.345741 10215 raft_consensus.cc:385] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.345849 10215 raft_consensus.cc:740] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.345700 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1062,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":160,"lbm_read_time_us":554,"lbm_reads_lt_1ms":25,"lbm_write_time_us":329,"lbm_writes_lt_1ms":9,"mutex_wait_us":26,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":2304,"update_count":1}
I20260807 07:53:19.346168 10215 consensus_queue.cc:260] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.346413 10215 raft_consensus.cc:399] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.346447 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 486 bytes on disk
I20260807 07:53:19.346583 10215 raft_consensus.cc:493] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.346875 10215 raft_consensus.cc:3060] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.347285 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":129,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.348498 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.243675
I20260807 07:53:19.350247 10215 raft_consensus.cc:515] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.350589 10215 leader_election.cc:304] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.350903 10215 leader_election.cc:290] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.350989 10219 raft_consensus.cc:2804] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.351195 10219 raft_consensus.cc:697] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.351514 10219 consensus_queue.cc:237] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.352617 10215 ts_tablet_manager.cc:1434] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 07:53:19.361037 9886 catalog_manager.cc:5697] T a759131412044e27b064cc4401c14d4a P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.392225 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.032s sys 0.000s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":32939,"cfile_cache_miss":28,"cfile_cache_miss_bytes":49253,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":765,"dirs.run_cpu_time_us":236,"dirs.run_wall_time_us":1254,"drs_written":1,"lbm_read_time_us":13490,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":59,"lbm_write_time_us":1201,"lbm_writes_lt_1ms":32,"mutex_wait_us":117,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":173,"update_count":1}
I20260807 07:53:19.393236 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:19.398949 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.030s sys 0.010s Metrics: {"bytes_written":390,"cfile_cache_hit":14,"cfile_cache_hit_bytes":32969,"cfile_cache_miss":34,"cfile_cache_miss_bytes":49723,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":1077,"dirs.run_cpu_time_us":1112,"dirs.run_wall_time_us":6794,"drs_written":1,"lbm_read_time_us":1589,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1467,"lbm_writes_lt_1ms":32,"mutex_wait_us":127,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":173,"spinlock_wait_cycles":12416,"thread_start_us":2616,"threads_started":1,"update_count":2}
I20260807 07:53:19.399840 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7763 bytes on disk
I20260807 07:53:19.400415 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.400611 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.001s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:19.401089 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:19.406759 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":81,"lbm_read_time_us":151,"lbm_reads_lt_1ms":8,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:19.407351 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7763 bytes on disk
I20260807 07:53:19.407931 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.432209 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34052:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.445108 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f9eb3af5b25e4aa39b56446ef88c5150 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.86 [id=572bbb00d3fc431c85fa1c7e05eb7738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.445690 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9eb3af5b25e4aa39b56446ef88c5150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.451844 10215 tablet_bootstrap.cc:492] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.454361 10215 tablet_bootstrap.cc:654] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.457847 10215 tablet_bootstrap.cc:492] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.458065 10215 ts_tablet_manager.cc:1403] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:19.459300 10215 raft_consensus.cc:359] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.459550 10215 raft_consensus.cc:385] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.459755 10215 raft_consensus.cc:740] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.460081 10215 consensus_queue.cc:260] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.460381 10215 raft_consensus.cc:399] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.460556 10215 raft_consensus.cc:493] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.460795 10215 raft_consensus.cc:3060] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.463667 10215 raft_consensus.cc:515] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.464005 10215 leader_election.cc:304] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.464272 10215 leader_election.cc:290] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.464341 10219 raft_consensus.cc:2804] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.464521 10219 raft_consensus.cc:697] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.464840 10219 consensus_queue.cc:237] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.465016 10215 ts_tablet_manager.cc:1434] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260807 07:53:19.475095 9886 catalog_manager.cc:5697] T f9eb3af5b25e4aa39b56446ef88c5150 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.517648 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34066:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.523896 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260807 07:53:19.531342 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 72705090a07a4eb387797a8c992c6ae2 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.87 [id=c3a6b7569567462bad4369b9939d14db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.532074 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72705090a07a4eb387797a8c992c6ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.538020 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":1651,"drs_written":1,"lbm_read_time_us":291,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1006,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:19.538937 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 480 bytes on disk
I20260807 07:53:19.539546 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.540117 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.247883
I20260807 07:53:19.540614 10215 tablet_bootstrap.cc:492] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.542634 10215 tablet_bootstrap.cc:654] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.546625 10215 tablet_bootstrap.cc:492] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.546859 10215 ts_tablet_manager.cc:1403] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:19.547830 10215 raft_consensus.cc:359] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.548146 10215 raft_consensus.cc:385] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.548298 10215 raft_consensus.cc:740] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.548727 10215 consensus_queue.cc:260] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.548964 10215 raft_consensus.cc:399] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.549156 10215 raft_consensus.cc:493] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.549319 10215 raft_consensus.cc:3060] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.552147 10215 raft_consensus.cc:515] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.552484 10215 leader_election.cc:304] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.552790 10215 leader_election.cc:290] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.552951 10219 raft_consensus.cc:2804] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.553123 10219 raft_consensus.cc:697] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.553634 10215 ts_tablet_manager.cc:1434] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 07:53:19.553539 10219 consensus_queue.cc:237] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.562191 9886 catalog_manager.cc:5697] T 72705090a07a4eb387797a8c992c6ae2 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.569048 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.023s sys 0.003s Metrics: {"bytes_written":44195,"cfile_cache_hit":13,"cfile_cache_hit_bytes":33599,"cfile_cache_miss":31,"cfile_cache_miss_bytes":50573,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":4392,"drs_written":1,"lbm_read_time_us":996,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1329,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":177}
I20260807 07:53:19.569870 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7934 bytes on disk
I20260807 07:53:19.570539 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.606314 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34068:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.619454 10099 tablet_service.cc:1511] Processing CreateTablet for tablet e54da0253ea840c1abcf75e02acf891e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.88 [id=9af18658aaf7469481ebb23cd2b9f338]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.620035 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54da0253ea840c1abcf75e02acf891e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.625643 10215 tablet_bootstrap.cc:492] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.628227 10215 tablet_bootstrap.cc:654] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.631912 10215 tablet_bootstrap.cc:492] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.632157 10215 ts_tablet_manager.cc:1403] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260807 07:53:19.633185 10215 raft_consensus.cc:359] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.633433 10215 raft_consensus.cc:385] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.633585 10215 raft_consensus.cc:740] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.633877 10215 consensus_queue.cc:260] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.634135 10215 raft_consensus.cc:399] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.634274 10215 raft_consensus.cc:493] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.634467 10215 raft_consensus.cc:3060] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.637769 10215 raft_consensus.cc:515] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.638139 10215 leader_election.cc:304] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.638463 10215 leader_election.cc:290] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.638585 10219 raft_consensus.cc:2804] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.638940 10219 raft_consensus.cc:697] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.639235 10215 ts_tablet_manager.cc:1434] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260807 07:53:19.639289 10219 consensus_queue.cc:237] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.647291 9886 catalog_manager.cc:5697] T e54da0253ea840c1abcf75e02acf891e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.651396 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000089
I20260807 07:53:19.658950 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000088
I20260807 07:53:19.673954 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.008s sys 0.004s Metrics: {"bytes_written":6069,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1335,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1260,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:19.674865 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.244590
I20260807 07:53:19.680507 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.018s sys 0.004s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":965,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4837,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:19.681473 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249688
I20260807 07:53:19.710130 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.023s sys 0.008s Metrics: {"bytes_written":44659,"cfile_cache_hit":13,"cfile_cache_hit_bytes":34088,"cfile_cache_miss":31,"cfile_cache_miss_bytes":51386,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":2982,"dirs.run_cpu_time_us":990,"dirs.run_wall_time_us":5356,"drs_written":1,"lbm_read_time_us":2190,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1713,"lbm_writes_lt_1ms":28,"mutex_wait_us":1256,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":179}
I20260807 07:53:19.712673 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8041 bytes on disk
I20260807 07:53:19.714219 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.714936 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34082:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.726068 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.035s sys 0.001s Metrics: {"bytes_written":44659,"cfile_cache_hit":14,"cfile_cache_hit_bytes":34085,"cfile_cache_miss":34,"cfile_cache_miss_bytes":51765,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":2127,"dirs.run_cpu_time_us":1023,"dirs.run_wall_time_us":5354,"drs_written":1,"lbm_read_time_us":1316,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1048,"lbm_writes_lt_1ms":28,"mutex_wait_us":777,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":179}
I20260807 07:53:19.726835 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8041 bytes on disk
I20260807 07:53:19.727432 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.728020 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:19.729920 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 25aa83cb89214be2bf8fd18b8984f409 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.89 [id=41612d8c9c8e4d50a28c119c18777e9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.730386 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25aa83cb89214be2bf8fd18b8984f409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.735954 10215 tablet_bootstrap.cc:492] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.737607 10215 tablet_bootstrap.cc:654] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.739766 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":1060,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":861,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:19.740482 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.541872
I20260807 07:53:19.741194 10215 tablet_bootstrap.cc:492] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.741485 10215 ts_tablet_manager.cc:1403] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:19.742883 10215 raft_consensus.cc:359] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.743193 10215 raft_consensus.cc:385] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.743311 10215 raft_consensus.cc:740] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.743628 10215 consensus_queue.cc:260] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.743835 10215 raft_consensus.cc:399] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.743983 10215 raft_consensus.cc:493] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.744096 10215 raft_consensus.cc:3060] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.746609 10215 raft_consensus.cc:515] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.746927 10215 leader_election.cc:304] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.747365 10215 leader_election.cc:290] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.747386 10219 raft_consensus.cc:2804] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.747620 10219 raft_consensus.cc:697] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.748108 10219 consensus_queue.cc:237] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.749112 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.004s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":460,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":88,"lbm_read_time_us":381,"lbm_reads_lt_1ms":25,"lbm_write_time_us":366,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:19.752156 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.196372
I20260807 07:53:19.754555 9886 catalog_manager.cc:5697] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.749020 10215 ts_tablet_manager.cc:1434] T 25aa83cb89214be2bf8fd18b8984f409 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260807 07:53:19.787596 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.025s sys 0.004s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":34456,"cfile_cache_miss":31,"cfile_cache_miss_bytes":51278,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1078,"drs_written":1,"lbm_read_time_us":1938,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1653,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":181,"update_count":2}
I20260807 07:53:19.788614 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8107 bytes on disk
I20260807 07:53:19.789806 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.818151 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34088:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.821646 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:19.830570 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 432cf455719f4aabad1564f14aff7518 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.90 [id=44b16b32e6814972acd4afa6d2fe20c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.831187 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 432cf455719f4aabad1564f14aff7518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.837250 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.006s sys 0.004s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":4666,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1269,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:19.838229 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 466 bytes on disk
I20260807 07:53:19.838874 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.839293 10215 tablet_bootstrap.cc:492] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.839680 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.236668
I20260807 07:53:19.842253 10215 tablet_bootstrap.cc:654] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.846071 10215 tablet_bootstrap.cc:492] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.846282 10215 ts_tablet_manager.cc:1403] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260807 07:53:19.847254 10215 raft_consensus.cc:359] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.847474 10215 raft_consensus.cc:385] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.847673 10215 raft_consensus.cc:740] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.848021 10215 consensus_queue.cc:260] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.848351 10215 raft_consensus.cc:399] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.848503 10215 raft_consensus.cc:493] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.848758 10215 raft_consensus.cc:3060] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.851998 10215 raft_consensus.cc:515] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.852334 10215 leader_election.cc:304] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.852589 10215 leader_election.cc:290] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.852643 10219 raft_consensus.cc:2804] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.852910 10219 raft_consensus.cc:697] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.853331 10215 ts_tablet_manager.cc:1434] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260807 07:53:19.853291 10219 consensus_queue.cc:237] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.863271 9886 catalog_manager.cc:5697] T 432cf455719f4aabad1564f14aff7518 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.874307 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.024s sys 0.008s Metrics: {"bytes_written":45103,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34421,"cfile_cache_miss":28,"cfile_cache_miss_bytes":51461,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":405,"dirs.run_cpu_time_us":872,"dirs.run_wall_time_us":5469,"drs_written":1,"lbm_read_time_us":1381,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1300,"lbm_writes_lt_1ms":28,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":181,"spinlock_wait_cycles":14208}
I20260807 07:53:19.877738 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:19.887475 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.001s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1100,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":776,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:19.888150 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8501 bytes on disk
I20260807 07:53:19.888726 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":103,"lbm_reads_lt_1ms":8}
I20260807 07:53:19.889284 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.197596
I20260807 07:53:19.909436 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34092:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:19.925324 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 8f7f019d86ad49fcaabc10fd243aca4b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.91 [id=00de1df3a6d04bc390a86f7781991012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:19.925901 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f7f019d86ad49fcaabc10fd243aca4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:19.927187 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.028s sys 0.005s Metrics: {"bytes_written":45533,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34836,"cfile_cache_miss":28,"cfile_cache_miss_bytes":52074,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1240,"drs_written":1,"lbm_read_time_us":1416,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1242,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":65881,"rows_written":183,"spinlock_wait_cycles":2944}
I20260807 07:53:19.929806 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8227 bytes on disk
I20260807 07:53:19.931794 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.932510 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260807 07:53:19.933729 10215 tablet_bootstrap.cc:492] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:19.936355 10215 tablet_bootstrap.cc:654] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:19.940434 10215 tablet_bootstrap.cc:492] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:19.940934 10215 ts_tablet_manager.cc:1403] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 07:53:19.942224 10215 raft_consensus.cc:359] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.942482 10215 raft_consensus.cc:385] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:19.942610 10215 raft_consensus.cc:740] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:19.943296 10215 consensus_queue.cc:260] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.943629 10215 raft_consensus.cc:399] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:19.943883 10215 raft_consensus.cc:493] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:19.944150 10215 raft_consensus.cc:3060] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:19.946339 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.012s sys 0.000s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":1212,"drs_written":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1041,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:19.947074 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.199467
I20260807 07:53:19.947124 10215 raft_consensus.cc:515] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.947520 10215 leader_election.cc:304] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:19.947953 10215 leader_election.cc:290] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:19.948022 10219 raft_consensus.cc:2804] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:19.948343 10219 raft_consensus.cc:697] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:19.948773 10219 consensus_queue.cc:237] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:19.954414 10215 ts_tablet_manager.cc:1434] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.013s user 0.000s sys 0.008s
I20260807 07:53:19.965773 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:19.969866 9887 catalog_manager.cc:5697] T 8f7f019d86ad49fcaabc10fd243aca4b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:19.985045 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.012s sys 0.001s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":142,"dirs.run_wall_time_us":1148,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1137,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:19.986312 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 528 bytes on disk
I20260807 07:53:19.986459 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.026s sys 0.005s Metrics: {"bytes_written":45949,"cfile_cache_hit":12,"cfile_cache_hit_bytes":35163,"cfile_cache_miss":28,"cfile_cache_miss_bytes":52327,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":519,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":3368,"drs_written":1,"lbm_read_time_us":1315,"lbm_reads_lt_1ms":60,"lbm_write_time_us":5478,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":31,"num_input_rowsets":2,"peak_mem_usage":66073,"rows_written":185,"spinlock_wait_cycles":7936}
I20260807 07:53:19.987327 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:19.987910 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:19.988696 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.218747
I20260807 07:53:19.993244 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":172,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:19.993758 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8302 bytes on disk
I20260807 07:53:19.995098 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.025895 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.033s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":35159,"cfile_cache_miss":28,"cfile_cache_miss_bytes":52527,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":658,"dirs.run_cpu_time_us":1106,"dirs.run_wall_time_us":4996,"drs_written":1,"lbm_read_time_us":3521,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":2371,"lbm_writes_lt_1ms":32,"mutex_wait_us":55,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":185,"spinlock_wait_cycles":13568,"thread_start_us":774,"threads_started":1,"update_count":1}
I20260807 07:53:20.026885 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8319 bytes on disk
I20260807 07:53:20.027508 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.034719 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34104:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.042150 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:20.054924 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0d14be50d024487c9029bb53d6b0f880 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.92 [id=92532d2591b7410a85b2fbf789adef12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.054767 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.003s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":1334,"drs_written":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4,"lbm_write_time_us":895,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:20.055546 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d14be50d024487c9029bb53d6b0f880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.058091 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 471 bytes on disk
I20260807 07:53:20.058884 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.059665 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.211098
I20260807 07:53:20.062001 10215 tablet_bootstrap.cc:492] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.064347 10215 tablet_bootstrap.cc:654] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.068784 10215 tablet_bootstrap.cc:492] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.069365 10215 ts_tablet_manager.cc:1403] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 07:53:20.070686 10215 raft_consensus.cc:359] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.070942 10215 raft_consensus.cc:385] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.071084 10215 raft_consensus.cc:740] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.071400 10215 consensus_queue.cc:260] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.071671 10215 raft_consensus.cc:399] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.071825 10215 raft_consensus.cc:493] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.072023 10215 raft_consensus.cc:3060] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.074626 10215 raft_consensus.cc:515] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.074991 10215 leader_election.cc:304] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.075353 10215 leader_election.cc:290] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.075454 10219 raft_consensus.cc:2804] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.075721 10219 raft_consensus.cc:697] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.076045 10219 consensus_queue.cc:237] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.077041 10215 ts_tablet_manager.cc:1434] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260807 07:53:20.089082 9887 catalog_manager.cc:5697] T 0d14be50d024487c9029bb53d6b0f880 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.101593 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.026s sys 0.008s Metrics: {"bytes_written":45986,"cfile_cache_hit":13,"cfile_cache_hit_bytes":35195,"cfile_cache_miss":31,"cfile_cache_miss_bytes":53097,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":3323,"dirs.run_cpu_time_us":811,"dirs.run_wall_time_us":9730,"drs_written":1,"lbm_read_time_us":1453,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1633,"lbm_writes_lt_1ms":28,"mutex_wait_us":2337,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":185}
I20260807 07:53:20.102526 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260807 07:53:20.116310 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.001s Metrics: {"bytes_written":5192,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":1434,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":871,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:20.117377 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8694 bytes on disk
I20260807 07:53:20.118182 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":158,"lbm_reads_lt_1ms":8}
I20260807 07:53:20.118822 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191688
I20260807 07:53:20.148552 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.022s sys 0.004s Metrics: {"bytes_written":46406,"cfile_cache_hit":12,"cfile_cache_hit_bytes":35573,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53213,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":817,"dirs.run_cpu_time_us":1296,"dirs.run_wall_time_us":6485,"drs_written":1,"lbm_read_time_us":910,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1064,"lbm_writes_lt_1ms":28,"mutex_wait_us":40,"num_input_rowsets":2,"peak_mem_usage":66470,"rows_written":187}
I20260807 07:53:20.149772 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8433 bytes on disk
I20260807 07:53:20.150657 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.155292 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34120:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.170575 10099 tablet_service.cc:1511] Processing CreateTablet for tablet f16a1091d2174e1596668b3c8e9cca7f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.93 [id=c49b24be0d13473eb6e6e31c56df96d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.171305 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16a1091d2174e1596668b3c8e9cca7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.177007 10215 tablet_bootstrap.cc:492] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.180392 10215 tablet_bootstrap.cc:654] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.184891 10215 tablet_bootstrap.cc:492] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.185141 10215 ts_tablet_manager.cc:1403] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 07:53:20.186267 10215 raft_consensus.cc:359] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.186544 10215 raft_consensus.cc:385] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.186657 10215 raft_consensus.cc:740] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.186976 10215 consensus_queue.cc:260] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.187345 10215 raft_consensus.cc:399] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.187580 10215 raft_consensus.cc:493] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.187770 10215 raft_consensus.cc:3060] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.190999 10215 raft_consensus.cc:515] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.191309 10215 leader_election.cc:304] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.191744 10215 leader_election.cc:290] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.191887 10219 raft_consensus.cc:2804] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.192188 10219 raft_consensus.cc:697] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.192605 10219 consensus_queue.cc:237] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.192983 10215 ts_tablet_manager.cc:1434] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 07:53:20.202409 9887 catalog_manager.cc:5697] T f16a1091d2174e1596668b3c8e9cca7f P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.246368 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260807 07:53:20.248026 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34126:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.263792 10099 tablet_service.cc:1511] Processing CreateTablet for tablet bd5401357a6d49d79c36a8b4fcc82467 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.94 [id=09d50baa7c844816a5627000090b339f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.264395 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd5401357a6d49d79c36a8b4fcc82467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.271564 10215 tablet_bootstrap.cc:492] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.273545 10215 tablet_bootstrap.cc:654] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.273751 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.019s sys 0.004s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":1670,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1592,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1101,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:20.274767 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.232842
I20260807 07:53:20.278499 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:20.289201 10215 tablet_bootstrap.cc:492] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.289592 10215 ts_tablet_manager.cc:1403] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.007s
I20260807 07:53:20.290843 10215 raft_consensus.cc:359] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.291338 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.008s sys 0.002s Metrics: {"bytes_written":6032,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1230,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":928,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:20.292675 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.244999
I20260807 07:53:20.291136 10215 raft_consensus.cc:385] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.293148 10215 raft_consensus.cc:740] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.293529 10215 consensus_queue.cc:260] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.293825 10215 raft_consensus.cc:399] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.293993 10215 raft_consensus.cc:493] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.294169 10215 raft_consensus.cc:3060] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.298151 10215 raft_consensus.cc:515] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.298518 10215 leader_election.cc:304] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.298805 10219 raft_consensus.cc:2804] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.299069 10219 raft_consensus.cc:697] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.299345 10219 consensus_queue.cc:237] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.311033 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.033s sys 0.000s Metrics: {"bytes_written":46853,"cfile_cache_hit":13,"cfile_cache_hit_bytes":35935,"cfile_cache_miss":31,"cfile_cache_miss_bytes":54125,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1318,"dirs.run_cpu_time_us":653,"dirs.run_wall_time_us":3465,"drs_written":1,"lbm_read_time_us":1364,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1115,"lbm_writes_lt_1ms":28,"mutex_wait_us":143,"num_input_rowsets":2,"peak_mem_usage":66094,"rows_written":189,"spinlock_wait_cycles":16512}
I20260807 07:53:20.312346 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8513 bytes on disk
I20260807 07:53:20.312976 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.313632 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260807 07:53:20.316103 10215 leader_election.cc:290] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.321841 10215 ts_tablet_manager.cc:1434] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.032s user 0.003s sys 0.004s
I20260807 07:53:20.322116 9887 catalog_manager.cc:5697] T bd5401357a6d49d79c36a8b4fcc82467 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.329175 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.001s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":822,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":932,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":865,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:20.330464 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224796
I20260807 07:53:20.367072 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.031s sys 0.001s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36278,"cfile_cache_miss":28,"cfile_cache_miss_bytes":54026,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":669,"dirs.run_cpu_time_us":161,"dirs.run_wall_time_us":1127,"drs_written":1,"lbm_read_time_us":1331,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1485,"lbm_writes_lt_1ms":32,"mutex_wait_us":133,"num_input_rowsets":2,"peak_mem_usage":67091,"reinsert_count":0,"rows_written":191,"update_count":1}
I20260807 07:53:20.367995 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8594 bytes on disk
I20260807 07:53:20.367743 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.075s user 0.036s sys 0.008s Metrics: {"bytes_written":390,"cfile_cache_hit":13,"cfile_cache_hit_bytes":36305,"cfile_cache_miss":31,"cfile_cache_miss_bytes":54663,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1125,"dirs.run_cpu_time_us":1045,"dirs.run_wall_time_us":5659,"drs_written":1,"lbm_read_time_us":6825,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":66,"lbm_write_time_us":1703,"lbm_writes_lt_1ms":32,"mutex_wait_us":100,"num_input_rowsets":2,"peak_mem_usage":66265,"reinsert_count":0,"rows_written":191,"spinlock_wait_cycles":46080,"thread_start_us":7196,"threads_started":1,"update_count":2}
I20260807 07:53:20.368656 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8577 bytes on disk
I20260807 07:53:20.368672 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.369277 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.397110 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34142:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.402177 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260807 07:53:20.416481 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.012s sys 0.000s Metrics: {"bytes_written":5628,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":235,"dirs.run_wall_time_us":1435,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":826,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:20.417657 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 466 bytes on disk
I20260807 07:53:20.418202 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.418543 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 3dc73801992446b184237eaa1eb92c84 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.95 [id=0604e7d8f08d4ed89264b4428276b360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.418889 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238144
I20260807 07:53:20.419036 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc73801992446b184237eaa1eb92c84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.424724 10215 tablet_bootstrap.cc:492] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.426959 10215 tablet_bootstrap.cc:654] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.430152 10215 tablet_bootstrap.cc:492] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.430364 10215 ts_tablet_manager.cc:1403] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:20.431442 10215 raft_consensus.cc:359] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.431694 10215 raft_consensus.cc:385] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.431881 10215 raft_consensus.cc:740] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.432163 10215 consensus_queue.cc:260] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.432349 10215 raft_consensus.cc:399] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.432502 10215 raft_consensus.cc:493] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.432626 10215 raft_consensus.cc:3060] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.435099 10215 raft_consensus.cc:515] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.435406 10215 leader_election.cc:304] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.435853 10215 leader_election.cc:290] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.435889 10219 raft_consensus.cc:2804] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.436267 10219 raft_consensus.cc:697] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.436722 10219 consensus_queue.cc:237] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.437294 10215 ts_tablet_manager.cc:1434] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 07:53:20.446830 9886 catalog_manager.cc:5697] T 3dc73801992446b184237eaa1eb92c84 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.468226 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.031s sys 0.004s Metrics: {"bytes_written":47285,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36303,"cfile_cache_miss":28,"cfile_cache_miss_bytes":54347,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1095,"dirs.run_cpu_time_us":221,"dirs.run_wall_time_us":1071,"drs_written":1,"lbm_read_time_us":1153,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1504,"lbm_writes_lt_1ms":28,"mutex_wait_us":324,"num_input_rowsets":2,"peak_mem_usage":66899,"rows_written":191,"spinlock_wait_cycles":1920}
I20260807 07:53:20.469261 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8610 bytes on disk
I20260807 07:53:20.469897 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.470417 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260807 07:53:20.488543 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.016s sys 0.000s Metrics: {"bytes_written":5195,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1165,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":771,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:20.489885 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.198486
I20260807 07:53:20.498607 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34154:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.512859 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 9e5b4c1b117c4f9db16d2daaaddf1342 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.96 [id=7341d97214ac45f8b4c914264ffb4f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.513267 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e5b4c1b117c4f9db16d2daaaddf1342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.518594 10215 tablet_bootstrap.cc:492] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.521231 10215 tablet_bootstrap.cc:654] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.521380 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.029s sys 0.000s Metrics: {"bytes_written":47736,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36681,"cfile_cache_miss":28,"cfile_cache_miss_bytes":54867,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":988,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1103,"drs_written":1,"lbm_read_time_us":1489,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1178,"lbm_writes_lt_1ms":28,"mutex_wait_us":77,"num_input_rowsets":2,"peak_mem_usage":106637,"rows_written":193,"spinlock_wait_cycles":13696}
I20260807 07:53:20.522208 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8694 bytes on disk
I20260807 07:53:20.522944 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":133,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.523468 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260807 07:53:20.525776 10215 tablet_bootstrap.cc:492] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.525991 10215 ts_tablet_manager.cc:1403] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260807 07:53:20.527065 10215 raft_consensus.cc:359] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.527359 10215 raft_consensus.cc:385] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.527633 10215 raft_consensus.cc:740] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.528038 10215 consensus_queue.cc:260] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.528379 10215 raft_consensus.cc:399] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.528563 10215 raft_consensus.cc:493] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.528822 10215 raft_consensus.cc:3060] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.532322 10215 raft_consensus.cc:515] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.532619 10215 leader_election.cc:304] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.532895 10215 leader_election.cc:290] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.533002 10219 raft_consensus.cc:2804] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.533195 10219 raft_consensus.cc:697] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.533771 10219 consensus_queue.cc:237] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.533985 10215 ts_tablet_manager.cc:1434] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260807 07:53:20.536528 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.006s sys 0.004s Metrics: {"bytes_written":5158,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1139,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1209,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":22912}
I20260807 07:53:20.537408 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:20.538090 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.538936 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.194858
I20260807 07:53:20.540534 9887 catalog_manager.cc:5697] T 9e5b4c1b117c4f9db16d2daaaddf1342 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.580556 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.027s sys 0.008s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":37018,"cfile_cache_miss":28,"cfile_cache_miss_bytes":55136,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":548,"dirs.run_cpu_time_us":1053,"dirs.run_wall_time_us":5848,"drs_written":1,"lbm_read_time_us":1295,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1644,"lbm_writes_lt_1ms":32,"mutex_wait_us":157,"num_input_rowsets":2,"peak_mem_usage":106915,"reinsert_count":0,"rows_written":195,"spinlock_wait_cycles":1408,"update_count":1}
I20260807 07:53:20.582132 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8756 bytes on disk
I20260807 07:53:20.582870 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.595543 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34162:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.609705 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 36c879c77d724d6db918e39b90625640 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.97 [id=3b669f3b26a644009d865b039f10bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.610291 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36c879c77d724d6db918e39b90625640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.615262 10215 tablet_bootstrap.cc:492] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.616644 10215 tablet_bootstrap.cc:654] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.619560 10215 tablet_bootstrap.cc:492] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.619508 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:20.619777 10215 ts_tablet_manager.cc:1403] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:20.620301 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000091
I20260807 07:53:20.622810 10215 raft_consensus.cc:359] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.623167 10215 raft_consensus.cc:385] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.623291 10215 raft_consensus.cc:740] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.623625 10215 consensus_queue.cc:260] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.623826 10215 raft_consensus.cc:399] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.624050 10215 raft_consensus.cc:493] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.624224 10215 raft_consensus.cc:3060] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.628305 10215 raft_consensus.cc:515] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.628620 10215 leader_election.cc:304] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.628899 10215 leader_election.cc:290] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.629010 10219 raft_consensus.cc:2804] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.629220 10219 raft_consensus.cc:697] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.629565 10215 ts_tablet_manager.cc:1434] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260807 07:53:20.629635 10219 consensus_queue.cc:237] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.633316 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.000s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1163,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1265,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:20.633816 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.007s Metrics: {"bytes_written":6031,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1250,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":980,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:20.634819 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 508 bytes on disk
I20260807 07:53:20.635581 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.638164 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.216555
I20260807 07:53:20.639227 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.216554
I20260807 07:53:20.655407 9887 catalog_manager.cc:5697] T 36c879c77d724d6db918e39b90625640 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.680061 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.031s sys 0.001s Metrics: {"bytes_written":48564,"cfile_cache_hit":13,"cfile_cache_hit_bytes":37410,"cfile_cache_miss":31,"cfile_cache_miss_bytes":56266,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":510,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":1705,"drs_written":1,"lbm_read_time_us":1489,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1250,"lbm_writes_lt_1ms":28,"mutex_wait_us":27,"num_input_rowsets":2,"peak_mem_usage":107214,"rows_written":197}
I20260807 07:53:20.683385 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000006
I20260807 07:53:20.685109 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.031s sys 0.004s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":37376,"cfile_cache_miss":31,"cfile_cache_miss_bytes":56304,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1127,"dirs.run_cpu_time_us":489,"dirs.run_wall_time_us":2954,"drs_written":1,"lbm_read_time_us":1409,"lbm_reads_lt_1ms":63,"lbm_write_time_us":4782,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"mutex_wait_us":94,"num_input_rowsets":2,"peak_mem_usage":107247,"reinsert_count":0,"rows_written":197,"spinlock_wait_cycles":4352,"update_count":1}
I20260807 07:53:20.686185 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8846 bytes on disk
I20260807 07:53:20.686710 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.688215 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:20.688735 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8846 bytes on disk
I20260807 07:53:20.689882 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":120,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.712251 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34168:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.726454 10099 tablet_service.cc:1511] Processing CreateTablet for tablet d12eccaa7a144dc4a11e71aa3087d8bd (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.98 [id=3937e44e660344ffa8086069ff152cdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.726950 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12eccaa7a144dc4a11e71aa3087d8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.731447 10215 tablet_bootstrap.cc:492] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.733074 10215 tablet_bootstrap.cc:654] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.735884 10215 tablet_bootstrap.cc:492] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.736030 10215 ts_tablet_manager.cc:1403] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:20.736697 10215 raft_consensus.cc:359] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.736842 10215 raft_consensus.cc:385] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.736966 10215 raft_consensus.cc:740] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.737196 10215 consensus_queue.cc:260] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.737424 10215 raft_consensus.cc:399] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.737541 10215 raft_consensus.cc:493] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.737649 10215 raft_consensus.cc:3060] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.739807 10215 raft_consensus.cc:515] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.740130 10215 leader_election.cc:304] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.740511 10215 leader_election.cc:290] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.740521 11009 raft_consensus.cc:2804] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.740767 11009 raft_consensus.cc:697] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.741035 10215 ts_tablet_manager.cc:1434] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260807 07:53:20.741062 11009 consensus_queue.cc:237] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.747908 9886 catalog_manager.cc:5697] T d12eccaa7a144dc4a11e71aa3087d8bd P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.791767 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34182:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.806250 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 002ee18e00b64050995c8595af3c267e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.99 [id=aa9cbd0aca684beab9b1372979849c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.806728 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002ee18e00b64050995c8595af3c267e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.811522 10215 tablet_bootstrap.cc:492] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.813647 10215 tablet_bootstrap.cc:654] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.817238 10215 tablet_bootstrap.cc:492] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.817452 10215 ts_tablet_manager.cc:1403] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 07:53:20.818151 10215 raft_consensus.cc:359] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.818367 10215 raft_consensus.cc:385] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.818503 10215 raft_consensus.cc:740] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.818778 10215 consensus_queue.cc:260] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.818989 10215 raft_consensus.cc:399] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.819135 10215 raft_consensus.cc:493] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.819307 10215 raft_consensus.cc:3060] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.822470 10215 raft_consensus.cc:515] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.822774 10215 leader_election.cc:304] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.823092 10215 leader_election.cc:290] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.823091 11009 raft_consensus.cc:2804] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.823477 11009 raft_consensus.cc:697] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.823755 10215 ts_tablet_manager.cc:1434] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260807 07:53:20.823752 11009 consensus_queue.cc:237] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.832636 9887 catalog_manager.cc:5697] T 002ee18e00b64050995c8595af3c267e P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.833899 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260807 07:53:20.855057 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.005s sys 0.007s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":3703,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1198,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:20.855808 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 506 bytes on disk
I20260807 07:53:20.856477 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.857002 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.245314
I20260807 07:53:20.876546 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34196:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:20.892177 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 34986a4cf7ce460892e17f8e22fd90db (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.100 [id=0291e8d36bb64061a412203f5429a5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:20.892715 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34986a4cf7ce460892e17f8e22fd90db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:20.898691 10215 tablet_bootstrap.cc:492] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:20.899076 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.033s sys 0.004s Metrics: {"bytes_written":49481,"cfile_cache_hit":14,"cfile_cache_hit_bytes":38150,"cfile_cache_miss":34,"cfile_cache_miss_bytes":57728,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":1227,"dirs.run_cpu_time_us":714,"dirs.run_wall_time_us":4910,"drs_written":1,"lbm_read_time_us":2086,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1167,"lbm_writes_lt_1ms":28,"mutex_wait_us":135,"num_input_rowsets":2,"peak_mem_usage":108432,"rows_written":201,"spinlock_wait_cycles":3328,"thread_start_us":351,"threads_started":1}
I20260807 07:53:20.900204 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260807 07:53:20.901190 10215 tablet_bootstrap.cc:654] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:20.904987 10215 tablet_bootstrap.cc:492] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:20.905176 10215 ts_tablet_manager.cc:1403] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 07:53:20.906232 10215 raft_consensus.cc:359] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.906589 10215 raft_consensus.cc:385] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:20.906855 10215 raft_consensus.cc:740] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:20.907315 10215 consensus_queue.cc:260] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.907620 10215 raft_consensus.cc:399] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:20.907750 10215 raft_consensus.cc:493] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:20.907940 10215 raft_consensus.cc:3060] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:20.911546 10215 raft_consensus.cc:515] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.911880 10215 leader_election.cc:304] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:20.912140 10215 leader_election.cc:290] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:20.912297 11009 raft_consensus.cc:2804] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:20.912536 11009 raft_consensus.cc:697] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:20.912541 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":1043,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":985,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:20.912827 10215 ts_tablet_manager.cc:1434] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260807 07:53:20.912994 11009 consensus_queue.cc:237] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:20.913867 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9246 bytes on disk
I20260807 07:53:20.914755 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":193,"lbm_reads_lt_1ms":8}
I20260807 07:53:20.915313 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.194735
I20260807 07:53:20.922839 9887 catalog_manager.cc:5697] T 34986a4cf7ce460892e17f8e22fd90db P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:20.937671 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:20.945642 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:20.954021 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.008s sys 0.003s Metrics: {"bytes_written":6070,"cfile_init":1,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1169,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":916,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":1920}
I20260807 07:53:20.954965 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242256
I20260807 07:53:20.958451 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.010s sys 0.000s Metrics: {"bytes_written":6070,"cfile_init":1,"dirs.queue_time_us":89,"dirs.run_cpu_time_us":135,"dirs.run_wall_time_us":872,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":923,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:20.962196 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 518 bytes on disk
I20260807 07:53:20.962811 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.963850 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242256
I20260807 07:53:20.971837 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.029s sys 0.007s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":38499,"cfile_cache_miss":28,"cfile_cache_miss_bytes":57263,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":843,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":954,"drs_written":1,"lbm_read_time_us":1298,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1479,"lbm_writes_lt_1ms":32,"mutex_wait_us":48,"num_input_rowsets":2,"peak_mem_usage":108675,"reinsert_count":0,"rows_written":203,"spinlock_wait_cycles":3968,"update_count":1}
I20260807 07:53:20.972983 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9073 bytes on disk
I20260807 07:53:20.973692 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":133,"lbm_reads_lt_1ms":4}
I20260807 07:53:20.992376 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.033s sys 0.000s Metrics: {"bytes_written":49909,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38529,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57917,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":457,"dirs.run_cpu_time_us":833,"dirs.run_wall_time_us":5016,"drs_written":1,"lbm_read_time_us":1756,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1290,"lbm_writes_lt_1ms":28,"mutex_wait_us":20,"num_input_rowsets":2,"peak_mem_usage":108828,"rows_written":203,"spinlock_wait_cycles":41472}
I20260807 07:53:20.993440 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9088 bytes on disk
I20260807 07:53:20.994114 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.004209 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34210:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.009502 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.034s sys 0.003s Metrics: {"bytes_written":49909,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38529,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57917,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":888,"dirs.run_cpu_time_us":291,"dirs.run_wall_time_us":2301,"drs_written":1,"lbm_read_time_us":3426,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":2491,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"mutex_wait_us":40,"num_input_rowsets":2,"peak_mem_usage":108828,"rows_written":203,"spinlock_wait_cycles":5888}
I20260807 07:53:21.011445 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260807 07:53:21.019951 10099 tablet_service.cc:1511] Processing CreateTablet for tablet dca4e7ab0644429680efb01f29d206b0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.101 [id=93c078280b3e4c259d4afe478aca1572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.020525 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca4e7ab0644429680efb01f29d206b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.027693 10215 tablet_bootstrap.cc:492] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.029752 10215 tablet_bootstrap.cc:654] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.032965 10215 tablet_bootstrap.cc:492] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.033143 10215 ts_tablet_manager.cc:1403] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 07:53:21.033823 10215 raft_consensus.cc:359] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.034024 10215 raft_consensus.cc:385] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.034112 10215 raft_consensus.cc:740] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.034376 10215 consensus_queue.cc:260] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.034541 10215 raft_consensus.cc:399] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.034657 10215 raft_consensus.cc:493] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.034780 10215 raft_consensus.cc:3060] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.036687 10215 raft_consensus.cc:515] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.036929 10215 leader_election.cc:304] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.037178 10215 leader_election.cc:290] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.037257 11009 raft_consensus.cc:2804] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.037129 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.012s sys 0.008s Metrics: {"bytes_written":330,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":1171,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1317,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260807 07:53:21.037497 11009 raft_consensus.cc:697] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.037726 10215 ts_tablet_manager.cc:1434] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260807 07:53:21.037744 11009 consensus_queue.cc:237] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.038766 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.540984
I20260807 07:53:21.047844 9887 catalog_manager.cc:5697] T dca4e7ab0644429680efb01f29d206b0 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.051903 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":461,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":204,"lbm_read_time_us":464,"lbm_reads_lt_1ms":25,"lbm_write_time_us":322,"lbm_writes_lt_1ms":9,"mutex_wait_us":24,"peak_mem_usage":32768,"reinsert_count":0,"update_count":1}
I20260807 07:53:21.052695 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.221805
I20260807 07:53:21.087659 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34220:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.094269 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.032s sys 0.008s Metrics: {"bytes_written":50356,"cfile_cache_hit":13,"cfile_cache_hit_bytes":38898,"cfile_cache_miss":31,"cfile_cache_miss_bytes":57846,"cfile_init":10,"delta_iterators_relevant":6,"dirs.queue_time_us":7619,"dirs.run_cpu_time_us":1186,"dirs.run_wall_time_us":8746,"drs_written":1,"lbm_read_time_us":1274,"lbm_reads_lt_1ms":71,"lbm_write_time_us":948,"lbm_writes_lt_1ms":29,"num_input_rowsets":2,"peak_mem_usage":109076,"rows_written":205,"thread_start_us":8310,"threads_started":1}
I20260807 07:53:21.096468 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9254 bytes on disk
I20260807 07:53:21.097456 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.105576 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 8e8477d4bae64f7db5e10cd14a84e3e5 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.102 [id=db6c643179464c4da01ba9f5cef23074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.106065 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8477d4bae64f7db5e10cd14a84e3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.111002 10215 tablet_bootstrap.cc:492] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.112563 10215 tablet_bootstrap.cc:654] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.115258 10215 tablet_bootstrap.cc:492] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.115428 10215 ts_tablet_manager.cc:1403] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:21.116266 10215 raft_consensus.cc:359] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.116498 10215 raft_consensus.cc:385] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.116655 10215 raft_consensus.cc:740] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.116945 10215 consensus_queue.cc:260] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.117184 10215 raft_consensus.cc:399] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.117316 10215 raft_consensus.cc:493] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.117503 10215 raft_consensus.cc:3060] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.120059 10215 raft_consensus.cc:515] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.120399 10215 leader_election.cc:304] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.120807 10215 leader_election.cc:290] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.120865 11009 raft_consensus.cc:2804] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.121047 11009 raft_consensus.cc:697] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.121369 11009 consensus_queue.cc:237] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.121481 10215 ts_tablet_manager.cc:1434] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 07:53:21.128031 9887 catalog_manager.cc:5697] T 8e8477d4bae64f7db5e10cd14a84e3e5 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.172585 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34230:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.187073 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbebe05b3a241f4ac0365ebbefece0b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.103 [id=58bab50151544e31870cab5b94cdfe88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.187605 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbebe05b3a241f4ac0365ebbefece0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.196353 10215 tablet_bootstrap.cc:492] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.198289 10215 tablet_bootstrap.cc:654] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.201470 10215 tablet_bootstrap.cc:492] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.201711 10215 ts_tablet_manager.cc:1403] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 07:53:21.202381 10215 raft_consensus.cc:359] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.202615 10215 raft_consensus.cc:385] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.202711 10215 raft_consensus.cc:740] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.202997 10215 consensus_queue.cc:260] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.203173 10215 raft_consensus.cc:399] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.203274 10215 raft_consensus.cc:493] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.203369 10215 raft_consensus.cc:3060] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.205266 10215 raft_consensus.cc:515] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.205528 10215 leader_election.cc:304] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.205750 10215 leader_election.cc:290] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.205812 11009 raft_consensus.cc:2804] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.206067 11009 raft_consensus.cc:697] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.206336 10215 ts_tablet_manager.cc:1434] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 07:53:21.206333 11009 consensus_queue.cc:237] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.216495 9887 catalog_manager.cc:5697] T 3dbebe05b3a241f4ac0365ebbefece0b P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.224692 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:21.237042 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1309,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1123,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:21.237964 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 522 bytes on disk
I20260807 07:53:21.239279 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.241829 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238545
I20260807 07:53:21.245050 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260807 07:53:21.258025 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.012s sys 0.000s Metrics: {"bytes_written":6072,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1214,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1099,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:21.258904 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 522 bytes on disk
I20260807 07:53:21.259591 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.260728 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238730
I20260807 07:53:21.296639 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.049s sys 0.001s Metrics: {"bytes_written":51229,"cfile_cache_hit":13,"cfile_cache_hit_bytes":39637,"cfile_cache_miss":31,"cfile_cache_miss_bytes":59493,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1007,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1231,"drs_written":1,"lbm_read_time_us":1585,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1635,"lbm_writes_lt_1ms":29,"mutex_wait_us":49,"num_input_rowsets":2,"peak_mem_usage":110076,"rows_written":209}
I20260807 07:53:21.298189 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9535 bytes on disk
I20260807 07:53:21.299074 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":143,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.304373 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.032s sys 0.007s Metrics: {"bytes_written":51229,"cfile_cache_hit":12,"cfile_cache_hit_bytes":39640,"cfile_cache_miss":28,"cfile_cache_miss_bytes":59122,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":508,"dirs.run_cpu_time_us":203,"dirs.run_wall_time_us":1251,"drs_written":1,"lbm_read_time_us":1214,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1486,"lbm_writes_lt_1ms":29,"mutex_wait_us":136,"num_input_rowsets":2,"peak_mem_usage":110028,"rows_written":209}
I20260807 07:53:21.305244 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9535 bytes on disk
I20260807 07:53:21.308385 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.318516 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34240:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.104"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.333791 10099 tablet_service.cc:1511] Processing CreateTablet for tablet ea3f2d99b0db42c5a9c60a84fe5b6028 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.104 [id=87829255fd8f4172b418bd66570510e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.334375 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3f2d99b0db42c5a9c60a84fe5b6028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.340426 10215 tablet_bootstrap.cc:492] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.343376 10215 tablet_bootstrap.cc:654] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.347157 10215 tablet_bootstrap.cc:492] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.347283 10215 ts_tablet_manager.cc:1403] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260807 07:53:21.347886 10215 raft_consensus.cc:359] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.348023 10215 raft_consensus.cc:385] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.348100 10215 raft_consensus.cc:740] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.348344 10215 consensus_queue.cc:260] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.348650 10215 raft_consensus.cc:399] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.348800 10215 raft_consensus.cc:493] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.348795 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:21.349022 10215 raft_consensus.cc:3060] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.352263 10215 raft_consensus.cc:515] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.352512 10215 leader_election.cc:304] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.352762 10215 leader_election.cc:290] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.352963 11009 raft_consensus.cc:2804] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.353186 11009 raft_consensus.cc:697] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.353713 10215 ts_tablet_manager.cc:1434] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260807 07:53:21.353747 11009 consensus_queue.cc:237] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.361260 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":2918,"drs_written":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1008,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:21.363016 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 513 bytes on disk
I20260807 07:53:21.363191 9886 catalog_manager.cc:5697] T ea3f2d99b0db42c5a9c60a84fe5b6028 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.363647 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.364466 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.238270
I20260807 07:53:21.408152 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.028s sys 0.007s Metrics: {"bytes_written":337,"cfile_cache_hit":12,"cfile_cache_hit_bytes":39998,"cfile_cache_miss":30,"cfile_cache_miss_bytes":59758,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":309,"dirs.run_cpu_time_us":901,"dirs.run_wall_time_us":5643,"drs_written":1,"lbm_read_time_us":1195,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1334,"lbm_writes_lt_1ms":33,"num_input_rowsets":2,"peak_mem_usage":109903,"reinsert_count":0,"rows_written":211,"update_count":1}
I20260807 07:53:21.409384 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9603 bytes on disk
I20260807 07:53:21.410552 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":125,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.422227 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34248:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.435665 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 6d37178a90f34e049c077a99d1bd5151 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.105 [id=55d9ed1c594149faa92696bec3b45039]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.436218 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d37178a90f34e049c077a99d1bd5151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.442528 10215 tablet_bootstrap.cc:492] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.444752 10215 tablet_bootstrap.cc:654] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.448029 10215 tablet_bootstrap.cc:492] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.448323 10215 ts_tablet_manager.cc:1403] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 07:53:21.449434 10215 raft_consensus.cc:359] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.449681 10215 raft_consensus.cc:385] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.449822 10215 raft_consensus.cc:740] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.450138 10215 consensus_queue.cc:260] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.450325 10215 raft_consensus.cc:399] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.450500 10215 raft_consensus.cc:493] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.450661 10215 raft_consensus.cc:3060] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.453392 10215 raft_consensus.cc:515] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.453688 10215 leader_election.cc:304] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.454001 10215 leader_election.cc:290] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.454002 11009 raft_consensus.cc:2804] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.454214 11009 raft_consensus.cc:697] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.454509 11009 consensus_queue.cc:237] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.454769 10215 ts_tablet_manager.cc:1434] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260807 07:53:21.464278 9887 catalog_manager.cc:5697] T 6d37178a90f34e049c077a99d1bd5151 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.506244 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34260:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.106"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.525429 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 1fbac17eb5b0433ab54bff5f91c7e4c0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.106 [id=c574f0e5586348da97d3395df764ad56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.525957 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fbac17eb5b0433ab54bff5f91c7e4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.533381 10215 tablet_bootstrap.cc:492] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.535454 10215 tablet_bootstrap.cc:654] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.539798 10215 tablet_bootstrap.cc:492] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.540000 10215 ts_tablet_manager.cc:1403] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 07:53:21.541067 10215 raft_consensus.cc:359] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.541263 10215 raft_consensus.cc:385] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.541458 10215 raft_consensus.cc:740] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.541751 10215 consensus_queue.cc:260] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.542038 10215 raft_consensus.cc:399] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.542182 10215 raft_consensus.cc:493] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.542403 10215 raft_consensus.cc:3060] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.545123 10215 raft_consensus.cc:515] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.545492 10215 leader_election.cc:304] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.545903 10215 leader_election.cc:290] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.545980 11009 raft_consensus.cc:2804] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.546205 11009 raft_consensus.cc:697] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.546718 11009 consensus_queue.cc:237] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.547025 10215 ts_tablet_manager.cc:1434] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260807 07:53:21.550213 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:21.557004 9887 catalog_manager.cc:5697] T 1fbac17eb5b0433ab54bff5f91c7e4c0 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.559691 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:21.570827 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.014s sys 0.000s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":3505,"dirs.run_cpu_time_us":211,"dirs.run_wall_time_us":1459,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1544,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:21.571717 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242614
I20260807 07:53:21.587913 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.011s sys 0.001s Metrics: {"bytes_written":337,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":1111,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1380,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":6,"update_count":1}
I20260807 07:53:21.588857 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 507 bytes on disk
I20260807 07:53:21.589607 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.590174 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242614
I20260807 07:53:21.624614 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.044s sys 0.000s Metrics: {"bytes_written":52536,"cfile_cache_hit":13,"cfile_cache_hit_bytes":40774,"cfile_cache_miss":33,"cfile_cache_miss_bytes":61492,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1754,"dirs.run_cpu_time_us":472,"dirs.run_wall_time_us":2762,"drs_written":1,"lbm_read_time_us":2025,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1700,"lbm_writes_lt_1ms":29,"mutex_wait_us":868,"num_input_rowsets":2,"peak_mem_usage":110027,"rows_written":215}
I20260807 07:53:21.625694 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9789 bytes on disk
I20260807 07:53:21.630568 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.638003 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.036s sys 0.007s Metrics: {"bytes_written":52536,"cfile_cache_hit":13,"cfile_cache_hit_bytes":40774,"cfile_cache_miss":33,"cfile_cache_miss_bytes":61492,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1562,"dirs.run_cpu_time_us":545,"dirs.run_wall_time_us":3596,"drs_written":1,"lbm_read_time_us":1717,"lbm_reads_lt_1ms":65,"lbm_write_time_us":1666,"lbm_writes_lt_1ms":29,"mutex_wait_us":65,"num_input_rowsets":2,"peak_mem_usage":110027,"rows_written":215,"spinlock_wait_cycles":4864}
I20260807 07:53:21.638792 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9789 bytes on disk
I20260807 07:53:21.639492 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.648980 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34270:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.661505 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260807 07:53:21.662748 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 2e0832458ab24139a5b42b898fb49071 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.107 [id=ef618bf5378748da8b2ad29b1f9b6c53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.663246 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e0832458ab24139a5b42b898fb49071. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.668618 10215 tablet_bootstrap.cc:492] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.670210 10215 tablet_bootstrap.cc:654] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.673830 10215 tablet_bootstrap.cc:492] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.674016 10215 ts_tablet_manager.cc:1403] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 07:53:21.673998 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.001s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":243,"dirs.run_wall_time_us":1203,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":865,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":4864}
I20260807 07:53:21.674822 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.208621
I20260807 07:53:21.674836 10215 raft_consensus.cc:359] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.675036 10215 raft_consensus.cc:385] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.675254 10215 raft_consensus.cc:740] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.675601 10215 consensus_queue.cc:260] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.675848 10215 raft_consensus.cc:399] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.676034 10215 raft_consensus.cc:493] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.676203 10215 raft_consensus.cc:3060] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.678453 10215 raft_consensus.cc:515] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.678730 10215 leader_election.cc:304] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.679129 10215 leader_election.cc:290] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.679149 11009 raft_consensus.cc:2804] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.679359 11009 raft_consensus.cc:697] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.679621 11009 consensus_queue.cc:237] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.689859 9887 catalog_manager.cc:5697] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.680750 10215 ts_tablet_manager.cc:1434] T 2e0832458ab24139a5b42b898fb49071 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260807 07:53:21.716426 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.035s sys 0.004s Metrics: {"bytes_written":334,"cfile_cache_hit":13,"cfile_cache_hit_bytes":41103,"cfile_cache_miss":33,"cfile_cache_miss_bytes":61945,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1375,"dirs.run_cpu_time_us":1196,"dirs.run_wall_time_us":6041,"drs_written":1,"lbm_read_time_us":1258,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1500,"lbm_writes_lt_1ms":33,"mutex_wait_us":34,"num_input_rowsets":2,"peak_mem_usage":110017,"reinsert_count":0,"rows_written":217,"spinlock_wait_cycles":2688,"thread_start_us":1494,"threads_started":2,"update_count":1}
I20260807 07:53:21.717268 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:21.721781 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.004s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":106,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:21.724264 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9822 bytes on disk
I20260807 07:53:21.724776 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.725289 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:21.730758 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":72,"lbm_read_time_us":87,"lbm_reads_lt_1ms":8,"lbm_write_time_us":114,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:21.752393 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34282:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.108"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.766602 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 25d1044c382b428eadeadfca44d1d64d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.108 [id=48d2b9c1b1d642869a8806bfa3085f5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.767096 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25d1044c382b428eadeadfca44d1d64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.772738 10215 tablet_bootstrap.cc:492] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.774269 10215 tablet_bootstrap.cc:654] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.777686 10215 tablet_bootstrap.cc:492] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.777837 10215 ts_tablet_manager.cc:1403] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:21.778457 10215 raft_consensus.cc:359] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.778613 10215 raft_consensus.cc:385] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.778733 10215 raft_consensus.cc:740] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.778981 10215 consensus_queue.cc:260] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.779145 10215 raft_consensus.cc:399] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.779248 10215 raft_consensus.cc:493] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.779392 10215 raft_consensus.cc:3060] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.781477 10215 raft_consensus.cc:515] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.781674 10215 leader_election.cc:304] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.781852 10215 leader_election.cc:290] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.781944 11009 raft_consensus.cc:2804] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.782096 11009 raft_consensus.cc:697] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.782372 11009 consensus_queue.cc:237] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.782712 10215 ts_tablet_manager.cc:1434] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 07:53:21.789673 9887 catalog_manager.cc:5697] T 25d1044c382b428eadeadfca44d1d64d P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.829108 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34286:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.109"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.842310 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 183f29cf962a4f4c8ce6b6d6de7e2cdc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.109 [id=8903ced7c5ad49159c5192888eeb44df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.842849 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 183f29cf962a4f4c8ce6b6d6de7e2cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.848058 10215 tablet_bootstrap.cc:492] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.849766 10215 tablet_bootstrap.cc:654] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.852843 10215 tablet_bootstrap.cc:492] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.853063 10215 ts_tablet_manager.cc:1403] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 07:53:21.854151 10215 raft_consensus.cc:359] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.854657 10215 raft_consensus.cc:385] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.854799 10215 raft_consensus.cc:740] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.855134 10215 consensus_queue.cc:260] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.855369 10215 raft_consensus.cc:399] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.855547 10215 raft_consensus.cc:493] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.855752 10215 raft_consensus.cc:3060] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.858772 10215 raft_consensus.cc:515] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.859071 10215 leader_election.cc:304] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.859452 10215 leader_election.cc:290] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:21.859467 11009 raft_consensus.cc:2804] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.859709 11009 raft_consensus.cc:697] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.859975 11009 consensus_queue.cc:237] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.860272 10215 ts_tablet_manager.cc:1434] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260807 07:53:21.867327 9887 catalog_manager.cc:5697] T 183f29cf962a4f4c8ce6b6d6de7e2cdc P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:21.884788 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:21.890385 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260807 07:53:21.897123 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6071,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":276,"dirs.run_wall_time_us":1189,"drs_written":1,"lbm_read_time_us":127,"lbm_reads_lt_1ms":4,"lbm_write_time_us":968,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:21.897974 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207640
I20260807 07:53:21.904561 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":6071,"cfile_init":1,"dirs.queue_time_us":83,"dirs.run_cpu_time_us":260,"dirs.run_wall_time_us":1109,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1197,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":3456}
I20260807 07:53:21.905284 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.207640
I20260807 07:53:21.946693 9887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34288:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:21.963766 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.037s sys 0.008s Metrics: {"bytes_written":53846,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41885,"cfile_cache_miss":30,"cfile_cache_miss_bytes":62791,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":315,"dirs.run_cpu_time_us":888,"dirs.run_wall_time_us":4597,"drs_written":1,"lbm_read_time_us":1010,"lbm_reads_lt_1ms":62,"lbm_write_time_us":5841,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":110068,"rows_written":221}
I20260807 07:53:21.964749 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10064 bytes on disk
I20260807 07:53:21.964890 10099 tablet_service.cc:1511] Processing CreateTablet for tablet e16e1cfab3174f9d98b1a2938fc0cdab (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.110 [id=eb38ff3db84b4ce498af76de23052232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:21.965463 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e16e1cfab3174f9d98b1a2938fc0cdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:21.967092 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":1606,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260807 07:53:21.967697 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000018
I20260807 07:53:21.969759 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.072s user 0.042s sys 0.003s Metrics: {"bytes_written":53846,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41885,"cfile_cache_miss":30,"cfile_cache_miss_bytes":62791,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":914,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1556,"drs_written":1,"lbm_read_time_us":2401,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1574,"lbm_writes_lt_1ms":29,"mutex_wait_us":299,"num_input_rowsets":2,"peak_mem_usage":110068,"rows_written":221}
I20260807 07:53:21.970590 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000020
I20260807 07:53:21.974905 10215 tablet_bootstrap.cc:492] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:21.978266 10215 tablet_bootstrap.cc:654] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:21.981581 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:21.984867 10215 tablet_bootstrap.cc:492] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:21.985080 10215 ts_tablet_manager.cc:1403] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260807 07:53:21.985024 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.004s sys 0.006s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":68,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":935,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":931,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:21.986014 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10299 bytes on disk
I20260807 07:53:21.986395 10215 raft_consensus.cc:359] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.986698 10215 raft_consensus.cc:385] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:21.986925 10215 raft_consensus.cc:740] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:21.987069 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":242,"lbm_reads_lt_1ms":8}
I20260807 07:53:21.987285 10215 consensus_queue.cc:260] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.987579 10215 raft_consensus.cc:399] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:21.987703 10215 raft_consensus.cc:493] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:21.987886 10215 raft_consensus.cc:3060] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:21.987921 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191272
I20260807 07:53:21.989290 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.012s sys 0.001s Metrics: {"bytes_written":5159,"cfile_init":1,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":977,"drs_written":1,"lbm_read_time_us":308,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1216,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260807 07:53:21.990272 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260807 07:53:21.990972 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4}
I20260807 07:53:21.991756 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.191272
I20260807 07:53:21.992331 10215 raft_consensus.cc:515] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.992684 10215 leader_election.cc:304] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:21.993041 11009 raft_consensus.cc:2804] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:21.993208 11009 raft_consensus.cc:697] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:21.993508 11009 consensus_queue.cc:237] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:21.999511 10215 leader_election.cc:290] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:22.000393 10215 ts_tablet_manager.cc:1434] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.015s user 0.003s sys 0.006s
I20260807 07:53:22.018165 9887 catalog_manager.cc:5697] T e16e1cfab3174f9d98b1a2938fc0cdab P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:22.022660 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.022s sys 0.004s Metrics: {"bytes_written":6035,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":1156,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1397,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260807 07:53:22.023730 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.236802
I20260807 07:53:22.037533 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.033s sys 0.008s Metrics: {"bytes_written":54244,"cfile_cache_hit":12,"cfile_cache_hit_bytes":42228,"cfile_cache_miss":30,"cfile_cache_miss_bytes":63048,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1154,"dirs.run_cpu_time_us":341,"dirs.run_wall_time_us":2252,"drs_written":1,"lbm_read_time_us":1538,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1229,"lbm_writes_lt_1ms":29,"mutex_wait_us":684,"num_input_rowsets":2,"peak_mem_usage":108667,"rows_written":223}
I20260807 07:53:22.038475 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:22.044345 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.000s Metrics: {"bytes_written":334,"delete_count":0,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:22.044843 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10128 bytes on disk
I20260807 07:53:22.045792 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.054433 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.036s sys 0.002s Metrics: {"bytes_written":334,"cfile_cache_hit":12,"cfile_cache_hit_bytes":42228,"cfile_cache_miss":30,"cfile_cache_miss_bytes":63048,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":630,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":1074,"drs_written":1,"lbm_read_time_us":1506,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1570,"lbm_writes_lt_1ms":33,"mutex_wait_us":53,"num_input_rowsets":2,"peak_mem_usage":108667,"reinsert_count":0,"rows_written":223,"update_count":1}
I20260807 07:53:22.055430 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10128 bytes on disk
I20260807 07:53:22.055960 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.056500 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260807 07:53:22.061409 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"bytes_written":337,"delete_count":0,"lbm_write_time_us":176,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260807 07:53:22.061856 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260807 07:53:22.068451 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":469,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":52,"lbm_read_time_us":177,"lbm_reads_lt_1ms":8,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260807 07:53:22.075160 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.039s sys 0.000s Metrics: {"bytes_written":337,"cfile_cache_hit":13,"cfile_cache_hit_bytes":42221,"cfile_cache_miss":33,"cfile_cache_miss_bytes":63609,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":823,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":1012,"drs_written":1,"lbm_read_time_us":2704,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1489,"lbm_writes_lt_1ms":33,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":109916,"reinsert_count":0,"rows_written":223,"spinlock_wait_cycles":6016,"update_count":1}
I20260807 07:53:22.075994 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10114 bytes on disk
I20260807 07:53:22.076495 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.096081 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34290:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.111"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:22.110015 10099 tablet_service.cc:1511] Processing CreateTablet for tablet e97f53909f5d4387aad2b6962a4b4ba0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.111 [id=9cd4ff937eb649c9af45069813c9274a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:22.110486 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97f53909f5d4387aad2b6962a4b4ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:22.115056 10215 tablet_bootstrap.cc:492] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:22.116968 10215 tablet_bootstrap.cc:654] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:22.120307 10215 tablet_bootstrap.cc:492] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518: No bootstrap required, opened a new log
I20260807 07:53:22.120573 10215 ts_tablet_manager.cc:1403] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260807 07:53:22.121323 10215 raft_consensus.cc:359] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:22.121623 10215 raft_consensus.cc:385] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 07:53:22.121721 10215 raft_consensus.cc:740] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Initialized, Role: FOLLOWER
I20260807 07:53:22.122021 10215 consensus_queue.cc:260] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:22.122223 10215 raft_consensus.cc:399] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 07:53:22.122352 10215 raft_consensus.cc:493] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 07:53:22.122473 10215 raft_consensus.cc:3060] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Advancing to term 1
I20260807 07:53:22.125926 10215 raft_consensus.cc:515] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:22.126294 10215 leader_election.cc:304] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ecc5f2dc8874ab884abb1a8392e4518; no voters:
I20260807 07:53:22.126539 10215 leader_election.cc:290] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 07:53:22.126591 11009 raft_consensus.cc:2804] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 07:53:22.126904 11009 raft_consensus.cc:697] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecc5f2dc8874ab884abb1a8392e4518, State: Running, Role: LEADER
I20260807 07:53:22.127236 11009 consensus_queue.cc:237] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } }
I20260807 07:53:22.127447 10215 ts_tablet_manager.cc:1434] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 07:53:22.136758 9886 catalog_manager.cc:5697] T e97f53909f5d4387aad2b6962a4b4ba0 P 9ecc5f2dc8874ab884abb1a8392e4518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193). New cstate: current_term: 1 leader_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ecc5f2dc8874ab884abb1a8392e4518" member_type: VOTER last_known_addr { host: "127.2.169.193" port: 45819 } health_report { overall_health: HEALTHY } } }
I20260807 07:53:22.189808 9886 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34306:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/1.112"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 07:53:22.204717 10099 tablet_service.cc:1511] Processing CreateTablet for tablet 0ccf85259c5143bab933cd3c922e6471 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.112 [id=dffafac1403b4eda8976483122b7d32d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:22.205286 10099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ccf85259c5143bab933cd3c922e6471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:22.211860 10215 tablet_bootstrap.cc:492] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518: Bootstrap starting.
I20260807 07:53:22.213456 10215 tablet_bootstrap.cc:654] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518: Neither blocks nor log segments found. Creating new log.
W20260807 07:53:22.215334 10215 log.cc:1245] Error closing log: Illegal state: Log not open. State: 0
E20260807 07:53:22.216301 10215 ts_tablet_manager.cc:1425] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518: Tablet failed to bootstrap: IO error: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0/minicluster-data/ts-0/wal/wals/0ccf85259c5143bab933cd3c922e6471/.kudutmp.newsegmentRaGnI6 (524193792 bytes available vs 518351011 bytes reserved) (error 28)
I20260807 07:53:22.216468 10215 ts_tablet_manager.cc:1403] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 07:53:22.216709 10215 tablet_replica.cc:333] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518: stopping tablet replica
I20260807 07:53:22.216845 10215 raft_consensus.cc:2243] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:53:22.216967 10215 raft_consensus.cc:2272] T 0ccf85259c5143bab933cd3c922e6471 P 9ecc5f2dc8874ab884abb1a8392e4518 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 07:53:22.296864 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000087
I20260807 07:53:22.316618 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.014s sys 0.003s Metrics: {"bytes_written":5593,"cfile_init":1,"dirs.queue_time_us":110,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1508,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1184,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:22.317531 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.226445
I20260807 07:53:22.320824 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000092
I20260807 07:53:22.329622 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.008s sys 0.000s Metrics: {"bytes_written":5593,"cfile_init":1,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":191,"dirs.run_wall_time_us":1205,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":621,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:22.330400 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.226421
I20260807 07:53:22.330401 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260807 07:53:22.340485 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.004s Metrics: {"bytes_written":5593,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":232,"dirs.run_wall_time_us":1148,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":826,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260807 07:53:22.341316 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 427 bytes on disk
I20260807 07:53:22.344693 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.345261 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.226435
I20260807 07:53:22.349758 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.023s sys 0.005s Metrics: {"bytes_written":55134,"cfile_cache_hit":13,"cfile_cache_hit_bytes":42955,"cfile_cache_miss":33,"cfile_cache_miss_bytes":64681,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1249,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":1011,"drs_written":1,"lbm_read_time_us":1067,"lbm_reads_lt_1ms":65,"lbm_write_time_us":918,"lbm_writes_lt_1ms":29,"mutex_wait_us":196,"num_input_rowsets":2,"peak_mem_usage":108899,"rows_written":227}
I20260807 07:53:22.350538 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10289 bytes on disk
I20260807 07:53:22.351125 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.360112 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.028s sys 0.000s Metrics: {"bytes_written":55134,"cfile_cache_hit":13,"cfile_cache_hit_bytes":42955,"cfile_cache_miss":33,"cfile_cache_miss_bytes":64793,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":909,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":950,"drs_written":1,"lbm_read_time_us":1191,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1043,"lbm_writes_lt_1ms":29,"mutex_wait_us":218,"num_input_rowsets":2,"peak_mem_usage":108899,"rows_written":227,"spinlock_wait_cycles":768}
I20260807 07:53:22.360852 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10289 bytes on disk
I20260807 07:53:22.361325 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
I20260807 07:53:22.370671 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.021s sys 0.003s Metrics: {"bytes_written":55134,"cfile_cache_hit":13,"cfile_cache_hit_bytes":42989,"cfile_cache_miss":33,"cfile_cache_miss_bytes":64693,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":826,"dirs.run_cpu_time_us":134,"dirs.run_wall_time_us":954,"drs_written":1,"lbm_read_time_us":1077,"lbm_reads_lt_1ms":65,"lbm_write_time_us":1308,"lbm_writes_lt_1ms":29,"mutex_wait_us":102,"num_input_rowsets":2,"peak_mem_usage":108865,"rows_written":227,"spinlock_wait_cycles":896}
I20260807 07:53:22.371493 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10289 bytes on disk
I20260807 07:53:22.371867 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4}
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/dynamic_multi_master-test.cc:372: Failure
Failed
Bad status: Timed out: Timed out waiting for Table Creation
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/dynamic_multi_master-test.cc:1082: Failure
Expected: StartClusterWithSysCatalogGCed( &master_hps, {"--consensus_rpc_timeout_ms=2000", "--follower_unavailable_considered_failed_sec=4"}) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260807 07:53:52.142256 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 10009
I20260807 07:53:52.158552 10172 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 07:53:52.235198 9961 catalog_manager.cc:6160] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Tablet 0ccf85259c5143bab933cd3c922e6471 (table Table.TestRecoverDeadMasterSysCatalogCopy/1.112 [id=dffafac1403b4eda8976483122b7d32d]) was not created within the allowed timeout. Replacing with a new tablet ef91c395f93b4fc4b2a653959db088ec
I20260807 07:53:52.251295 9961 catalog_manager.cc:5984] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6: Sending DeleteTablet for 1 replicas of tablet 0ccf85259c5143bab933cd3c922e6471
I20260807 07:53:52.401682 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008357
I20260807 07:53:52.406034 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008355
I20260807 07:53:52.414096 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.006s Metrics: {"bytes_written":5001,"cfile_init":1,"dirs.queue_time_us":441,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":804,"drs_written":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4,"lbm_write_time_us":902,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":277,"threads_started":1}
I20260807 07:53:52.415009 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260807 07:53:52.415515 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260807 07:53:52.415653 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008354
I20260807 07:53:52.415781 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"bytes_written":5001,"cfile_init":1,"dirs.queue_time_us":363,"dirs.run_cpu_time_us":131,"dirs.run_wall_time_us":769,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":742,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"spinlock_wait_cycles":896,"thread_start_us":267,"threads_started":1}
I20260807 07:53:52.416117 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014981
I20260807 07:53:52.416579 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014981
I20260807 07:53:52.440444 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.009s sys 0.012s Metrics: {"bytes_written":5001,"cfile_init":1,"dirs.queue_time_us":12497,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":603,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":984,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":12340,"threads_started":1}
I20260807 07:53:52.441486 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260807 07:53:52.442143 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260807 07:53:52.442848 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014981
I20260807 07:53:52.466892 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.040s sys 0.008s Metrics: {"bytes_written":55414,"cfile_cache_hit":12,"cfile_cache_hit_bytes":43176,"cfile_cache_miss":30,"cfile_cache_miss_bytes":64340,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1282,"dirs.run_cpu_time_us":773,"dirs.run_wall_time_us":3245,"drs_written":1,"lbm_read_time_us":1530,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1156,"lbm_writes_lt_1ms":29,"mutex_wait_us":18,"num_input_rowsets":2,"peak_mem_usage":68787,"rows_written":228,"spinlock_wait_cycles":1408,"thread_start_us":10013,"threads_started":2}
I20260807 07:53:52.467742 9789 maintenance_manager.cc:419] P b1f16687ee13426cb547143f657d8e4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10347 bytes on disk
I20260807 07:53:52.468421 9788 maintenance_manager.cc:643] P b1f16687ee13426cb547143f657d8e4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260807 07:53:52.469515 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.033s sys 0.018s Metrics: {"bytes_written":55414,"cfile_cache_hit":12,"cfile_cache_hit_bytes":43176,"cfile_cache_miss":30,"cfile_cache_miss_bytes":64340,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":967,"dirs.run_cpu_time_us":782,"dirs.run_wall_time_us":3165,"drs_written":1,"lbm_read_time_us":1132,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1020,"lbm_writes_lt_1ms":29,"mutex_wait_us":118,"num_input_rowsets":2,"peak_mem_usage":68787,"rows_written":228,"thread_start_us":19901,"threads_started":2}
I20260807 07:53:52.470221 9702 maintenance_manager.cc:419] P 85836fd00d0b4e7f9dd80c844ee37519: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10347 bytes on disk
I20260807 07:53:52.470767 9701 maintenance_manager.cc:643] P 85836fd00d0b4e7f9dd80c844ee37519: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260807 07:53:52.480656 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.031s sys 0.004s Metrics: {"bytes_written":55414,"cfile_cache_hit":12,"cfile_cache_hit_bytes":43176,"cfile_cache_miss":30,"cfile_cache_miss_bytes":64340,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1164,"dirs.run_cpu_time_us":617,"dirs.run_wall_time_us":2668,"drs_written":1,"lbm_read_time_us":1669,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1299,"lbm_writes_lt_1ms":29,"mutex_wait_us":47,"num_input_rowsets":2,"peak_mem_usage":68787,"rows_written":228,"spinlock_wait_cycles":9984,"thread_start_us":990,"threads_started":3}
I20260807 07:53:52.481520 9875 maintenance_manager.cc:419] P e7af352b23b544d495c321f8c84767c6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10347 bytes on disk
I20260807 07:53:52.481966 9874 maintenance_manager.cc:643] P e7af352b23b544d495c321f8c84767c6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4}
I20260807 07:53:52.517201 10099 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ccf85259c5143bab933cd3c922e6471 with delete_type TABLET_DATA_DELETED (Replaced by ef91c395f93b4fc4b2a653959db088ec at 2026-08-07 07:53:52 UTC) from {username='slave'} at 127.0.0.1:42050
I20260807 07:53:52.517799 10098 tablet_service.cc:1511] Processing CreateTablet for tablet ef91c395f93b4fc4b2a653959db088ec (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/1.112 [id=dffafac1403b4eda8976483122b7d32d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 07:53:52.521586 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 10009
W20260807 07:53:52.576053 9873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.169.193:45819 (error 108)
I20260807 07:53:52.576593 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9680
I20260807 07:53:52.577992 9750 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 07:53:52.578591 9873 catalog_manager.cc:4755] TS 9ecc5f2dc8874ab884abb1a8392e4518 (127.2.169.193:45819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ccf85259c5143bab933cd3c922e6471: Network error: Client connection negotiation failed: client connection to 127.2.169.193:45819: connect: Connection refused (error 111)
I20260807 07:53:52.768800 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9680
W20260807 07:53:52.813400 9872 connection.cc:573] client connection to 127.2.169.254:42545 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 07:53:52.813894 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9760
W20260807 07:53:52.814729 9872 consensus_peers.cc:597] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 -> Peer 85836fd00d0b4e7f9dd80c844ee37519 (127.2.169.254:42545): Couldn't send request to peer 85836fd00d0b4e7f9dd80c844ee37519. Status: Network error: Client connection negotiation failed: client connection to 127.2.169.254:42545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 07:53:52.815147 9842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:53.005751 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9760
I20260807 07:53:53.051561 2727 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu pid 9851
I20260807 07:53:53.052807 9934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 07:53:53.263859 11624 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7af352b23b544d495c321f8c84767c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85836fd00d0b4e7f9dd80c844ee37519" member_type: VOTER last_known_addr { host: "127.2.169.254" port: 42545 } } peers { permanent_uuid: "b1f16687ee13426cb547143f657d8e4d" member_type: VOTER last_known_addr { host: "127.2.169.253" port: 45107 } } peers { permanent_uuid: "e7af352b23b544d495c321f8c84767c6" member_type: VOTER last_known_addr { host: "127.2.169.252" port: 40495 } } }
I20260807 07:53:53.264132 11624 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af352b23b544d495c321f8c84767c6 [sys.catalog]: This master's current role is: LEADER
I20260807 07:53:53.264588 2727 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu with pid 9851
2026-08-07T07:53:53Z chronyd exiting
I20260807 07:53:53.324151 2727 test_util.cc:182] -----------------------------------------------
I20260807 07:53:53.324275 2727 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_1.1786089128278131-2727-0
[ FAILED ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/1, where GetParam() = 3 (48664 ms)
[----------] 1 test from ParameterizedRecoverMasterTest (48664 ms total)
[----------] 1 test from ParameterizedAutoAddMasterTest
[ RUN ] ParameterizedAutoAddMasterTest.TestBasicAddition/2_orig_masters_secure
Loading random data
Initializing database '/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 07:53:53 dist-test-slave-bw88 krb5kdc[11659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 07:53:53 dist-test-slave-bw88 krb5kdc[11659](info): set up 2 sockets
Aug 07 07:53:53 dist-test-slave-bw88 krb5kdc[11659](info): commencing operation
krb5kdc: starting...
W20260807 07:53:55.615051 2727 mini_kdc.cc:121] Time spent starting KDC: real 2.286s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/test-user.keytab.
Aug 07 07:53:55 dist-test-slave-bw88 krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T07:53:55Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-07T07:53:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.169.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:53:56.109848 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:40583
--webserver_interface=127.2.169.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:40583,127.2.169.253:33193 with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 07:53:56.487251 11678 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:56.487586 11678 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:56.487749 11678 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:56.496088 11678 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:56.496204 11678 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:56.496311 11678 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:56.496382 11678 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:53:56.506982 11678 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:40583,127.2.169.253:33193
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.254:40583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.169.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:53:56.507344 11678 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:40583,127.2.169.253:33193'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:53:56.508708 11678 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:56.510221 11678 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:56.520007 11685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:56.519972 11684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:56.519747 11688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:56.520908 11678 server_base.cc:1061] running on GCE node
I20260807 07:53:56.521778 11678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:56.523156 11678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:56.524431 11678 hybrid_clock.cc:648] HybridClock initialized: now 1786089236524379 us; error 45 us; skew 500 ppm
I20260807 07:53:56.524767 11678 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:53:56 dist-test-slave-bw88 krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:53:56.528019 11678 init.cc:377] Logged in from keytab as kudu/127.2.169.254@KRBTEST.COM (short username kudu)
I20260807 07:53:56.529686 11678 webserver.cc:533] Webserver started at http://127.2.169.254:44685/ using document root <none> and password file <none>
I20260807 07:53:56.530524 11678 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:56.530651 11678 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:56.531042 11678 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:56.533878 11678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data/instance:
uuid: "4c96cea4ba9e4c7aa581f53b4efa589a"
format_stamp: "Formatted at 2026-08-07 07:53:56 on dist-test-slave-bw88"
I20260807 07:53:56.534754 11678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal/instance:
uuid: "4c96cea4ba9e4c7aa581f53b4efa589a"
format_stamp: "Formatted at 2026-08-07 07:53:56 on dist-test-slave-bw88"
I20260807 07:53:56.540525 11678 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 07:53:56.544307 11701 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:56.545735 11678 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 07:53:56.545951 11678 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal
uuid: "4c96cea4ba9e4c7aa581f53b4efa589a"
format_stamp: "Formatted at 2026-08-07 07:53:56 on dist-test-slave-bw88"
I20260807 07:53:56.546182 11678 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:56.576871 11678 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:56.580892 11678 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:56.581278 11678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:56.603392 11678 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.254:40583
I20260807 07:53:56.603415 11765 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.254:40583 every 8 connection(s)
I20260807 07:53:56.605077 11678 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/data/info.pb
I20260807 07:53:56.609670 11766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:56.612102 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 11678
I20260807 07:53:56.612358 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal/instance
I20260807 07:53:56.615875 11766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 40583 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 07:53:56 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
I20260807 07:53:56.640946 11766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } has no permanent_uuid. Determining permanent_uuid...
W20260807 07:53:56.642266 11703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
W20260807 07:53:56.644025 11766 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:33193: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.169.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.169.253@KRBTEST.COM" created.
I20260807 07:53:56.693449 11766 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } attempt: 1
W20260807 07:53:56.695788 11766 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:33193: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.169.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.2.169.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.2.169.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.169.253@KRBTEST.COM" created.
I20260807 07:53:56.764076 11766 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } attempt: 2
W20260807 07:53:56.766389 11766 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:33193: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.169.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.2.169.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab.
I20260807 07:53:56.781210 2727 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
/tmp/dist-test-taskDb4J1c/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:33193
--webserver_interface=127.2.169.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.2.169.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.169.254:40583,127.2.169.253:33193 with env {KRB5CCNAME=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 07:53:56.857702 11766 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } attempt: 3
W20260807 07:53:56.860085 11766 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:33193: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
I20260807 07:53:57.003405 11766 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } attempt: 4
W20260807 07:53:57.005760 11766 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.169.253:33193: Network error: Client connection negotiation failed: client connection to 127.2.169.253:33193: connect: Connection refused (error 111)
W20260807 07:53:57.151438 11777 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 07:53:57.151787 11777 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 07:53:57.151882 11777 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260807 07:53:57.160575 11777 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 07:53:57.160668 11777 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260807 07:53:57.160730 11777 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 07:53:57.160786 11777 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260807 07:53:57.171741 11777 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.169.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.169.254:40583,127.2.169.253:33193
--keytab_file=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/krb5kdc/kudu.keytab
--principal=kudu/127.2.169.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.169.253:33193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.169.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 07 Aug 2026 07:47:51 UTC on bdcb31816ec0
build id 13222
ASAN enabled
W20260807 07:53:57.172066 11777 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.169.254:40583,127.2.169.253:33193'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 07:53:57.173518 11777 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 07:53:57.174914 11777 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 07:53:57.183272 11786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:57.183398 11784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 07:53:57.184665 11788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 07:53:57.185263 11777 server_base.cc:1061] running on GCE node
I20260807 07:53:57.186093 11777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 07:53:57.187430 11777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 07:53:57.188660 11777 hybrid_clock.cc:648] HybridClock initialized: now 1786089237188589 us; error 42 us; skew 500 ppm
I20260807 07:53:57.188993 11777 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089237, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 07:53:57.192243 11777 init.cc:377] Logged in from keytab as kudu/127.2.169.253@KRBTEST.COM (short username kudu)
I20260807 07:53:57.193852 11777 webserver.cc:533] Webserver started at http://127.2.169.253:35347/ using document root <none> and password file <none>
I20260807 07:53:57.194648 11777 fs_manager.cc:362] Metadata directory not provided
I20260807 07:53:57.194770 11777 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 07:53:57.195147 11777 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 07:53:57.197737 11777 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data/instance:
uuid: "3655c1e9dce2449ba8cca1fdc7b93cd4"
format_stamp: "Formatted at 2026-08-07 07:53:57 on dist-test-slave-bw88"
I20260807 07:53:57.198511 11777 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal/instance:
uuid: "3655c1e9dce2449ba8cca1fdc7b93cd4"
format_stamp: "Formatted at 2026-08-07 07:53:57 on dist-test-slave-bw88"
I20260807 07:53:57.203979 11777 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 07:53:57.207603 11796 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 07:53:57.209024 11777 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 07:53:57.209215 11777 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data,/tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal
uuid: "3655c1e9dce2449ba8cca1fdc7b93cd4"
format_stamp: "Formatted at 2026-08-07 07:53:57 on dist-test-slave-bw88"
I20260807 07:53:57.209465 11777 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 07:53:57.237066 11777 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 07:53:57.246243 11777 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 07:53:57.246542 11777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 07:53:57.268460 11777 rpc_server.cc:307] RPC server started. Bound to: 127.2.169.253:33193
I20260807 07:53:57.268563 11863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.169.253:33193 every 8 connection(s)
I20260807 07:53:57.270061 11777 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/data/info.pb
I20260807 07:53:57.274415 11864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 07:53:57.278972 2727 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskDb4J1c/build/asan/bin/kudu as pid 11777
I20260807 07:53:57.279191 2727 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal/instance
I20260807 07:53:57.281586 11864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.254" port: 40583 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089237, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for kudu/127.2.169.254@KRBTEST.COM
I20260807 07:53:57.304975 11864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } has no permanent_uuid. Determining permanent_uuid...
I20260807 07:53:57.305136 11766 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.169.253" port: 33193 } attempt: 5
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.254@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
Aug 07 07:53:57 dist-test-slave-bw88 krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786089237, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.169.253@KRBTEST.COM for kudu/127.2.169.253@KRBTEST.COM
I20260807 07:53:57.327132 11766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a: Bootstrap starting.
I20260807 07:53:57.327997 11864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4: Bootstrap starting.
I20260807 07:53:57.330693 11766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:57.331077 11864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4: Neither blocks nor log segments found. Creating new log.
I20260807 07:53:57.331800 11766 log.cc:826] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a: Log is configured to *not* fsync() on all Append() calls
I20260807 07:53:57.332188 11864 log.cc:826] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4: Log is configured to *not* fsync() on all Append() calls
W20260807 07:53:57.333487 11766 log.cc:1245] Error closing log: Illegal state: Log not open. State: 0
I20260807 07:53:57.334501 11766 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a: stopping tablet replica
I20260807 07:53:57.334697 11766 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:53:57.334932 11766 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4c96cea4ba9e4c7aa581f53b4efa589a [term 0 FOLLOWER]: Raft consensus is shut down!
W20260807 07:53:57.335515 11864 log.cc:1245] Error closing log: Illegal state: Log not open. State: 0
E20260807 07:53:57.335814 11766 master.cc:417] Unable to init master catalog manager: IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmentHCobMZ (520781824 bytes available vs 518351011 bytes reserved) (error 28)
I20260807 07:53:57.336027 11678 master.cc:562] Master@127.2.169.254:40583 shutting down...
I20260807 07:53:57.336650 11864 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4: stopping tablet replica
I20260807 07:53:57.336828 11864 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 07:53:57.337019 11864 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3655c1e9dce2449ba8cca1fdc7b93cd4 [term 0 FOLLOWER]: Raft consensus is shut down!
E20260807 07:53:57.337811 11864 master.cc:417] Unable to init master catalog manager: IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmento5thfI (520781824 bytes available vs 518351011 bytes reserved) (error 28)
I20260807 07:53:57.338076 11777 master.cc:562] Master@127.2.169.253:33193 shutting down...
I20260807 07:53:57.365989 11777 master.cc:584] Master@127.2.169.253:33193 shutdown complete.
I20260807 07:53:57.366148 11678 master.cc:584] Master@127.2.169.254:40583 shutdown complete.
W20260807 07:53:57.368494 11777 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
W20260807 07:53:57.368489 11678 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-0/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmentHCobMZ (520781824 bytes available vs 518351011 bytes reserved) (error 28)IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskDb4J1c/test-tmp/dynamic_multi_master-test.1.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_secure.1786089128278131-2727-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmento5thfI (520781824 bytes available vs 518351011 bytes reserved) (error 28)
I20260807 07:53:57.373705 11678 logging.cc:436] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 4 messages since previous log ~0 seconds ago
F20260807 07:54:02.321135 2727 external_mini_cluster.cc:656] Check failed: leader_master() != nullptr Must have started at least 1 master before adding tablet servers
*** Check failure stack trace: ***
*** Aborted at 1786089242 (unix time) try "date -d @1786089242" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000aa7) received by PID 2727 (TID 0x7f9cda934d40) from PID 2727; stack trace: ***
@ 0x7f9ce686a980 (unknown) at ??:0
@ 0x7f9cdb3eee87 gsignal at ??:0
@ 0x7f9cdb3f07f1 abort at ??:0
@ 0x7f9cddeb6d5d google::LogMessage::Fail() at ??:0
@ 0x7f9cddebab23 google::LogMessage::SendToLog() at ??:0
@ 0x7f9cddeb675c google::LogMessage::Flush() at ??:0
@ 0x7f9cddeb7ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9cf4aff2a6 kudu::cluster::ExternalMiniCluster::AddTabletServer() at ??:0
@ 0x7f9cf4af886f kudu::cluster::ExternalMiniCluster::Start() at ??:0
@ 0x573ddc kudu::master::AutoAddMasterTest::SetUpWithTestArgs() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/dynamic_multi_master-test.cc:1443
@ 0x57487d kudu::master::ParameterizedAutoAddMasterTest::SetUp() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/dynamic_multi_master-test.cc:1799
@ 0x7f9cdfdf60ed testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7f9cdfdeab66 testing::Test::Run() at ??:0
@ 0x7f9cdfdead9d testing::TestInfo::Run() at ??:0
@ 0x7f9cdfdeb377 testing::TestSuite::Run() at ??:0
@ 0x7f9cdfdeb77c testing::internal::UnitTestImpl::RunAllTests() at ??:0
@ 0x7f9cdfdf660d testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7f9cdfdeae63 testing::UnitTest::Run() at ??:0
@ 0x5f3dd2 RUN_ALL_TESTS() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/gtest/gtest.h:2293
@ 0x5f1ca9 main at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115
@ 0x7f9cdb3d1c87 __libc_start_main at ??:0
@ 0x42ccba _start at ??:?