Note: This is test shard 1 of 6.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from DynamicMultiMasterTest
[ RUN ] DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr
2026-08-09T06:39:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:01Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 06:39:01.936893 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:34323
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_auto_join_cluster=false with env {}
W20260809 06:39:02.280295 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:02.280679 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:02.280838 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:02.288136 6597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:02.288277 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:02.288383 6597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:02.288497 6597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:02.297446 6597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:02.298790 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:02.300680 6597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:02.313424 6604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:02.313388 6605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:02.313409 6607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:02.313990 6597 server_base.cc:1034] running on GCE node
I20260809 06:39:02.314771 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:02.316123 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:02.317435 6597 hybrid_clock.cc:648] HybridClock initialized: now 1786257542317334 us; error 106 us; skew 500 ppm
I20260809 06:39:02.317771 6597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:02.324371 6597 webserver.cc:460] Webserver started at http://127.6.111.62:35045/ using document root <none> and password file <none>
I20260809 06:39:02.325080 6597 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:02.325232 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:02.325614 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:02.328320 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "26c9821b977247b183bbbbe3c332835c"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.329118 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "26c9821b977247b183bbbbe3c332835c"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.334336 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:02.337764 6616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:02.339012 6597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:39:02.339208 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "26c9821b977247b183bbbbe3c332835c"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.339416 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:02.360456 6597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:02.361279 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:02.361626 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:02.381528 6597 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:34323
I20260809 06:39:02.381548 6675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:34323 every 8 connection(s)
I20260809 06:39:02.383003 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:02.386204 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 6597
I20260809 06:39:02.386483 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:02.387253 6676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:02.397825 6676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c: Bootstrap starting.
I20260809 06:39:02.401067 6676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:02.402441 6676 log.cc:824] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:02.406111 6676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c: No bootstrap required, opened a new log
I20260809 06:39:02.415941 6676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER }
I20260809 06:39:02.416431 6676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:02.416594 6676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c9821b977247b183bbbbe3c332835c, State: Initialized, Role: FOLLOWER
I20260809 06:39:02.417285 6676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER }
I20260809 06:39:02.417639 6676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:02.417793 6676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:02.418004 6676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:02.420122 6676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER }
I20260809 06:39:02.420619 6676 leader_election.cc:302] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c9821b977247b183bbbbe3c332835c; no voters:
I20260809 06:39:02.421312 6676 leader_election.cc:288] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:02.421486 6681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:02.422370 6681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [term 1 LEADER]: Becoming Leader. State: Replica: 26c9821b977247b183bbbbe3c332835c, State: Running, Role: LEADER
I20260809 06:39:02.422966 6681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER }
I20260809 06:39:02.423640 6676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:02.426054 6683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c9821b977247b183bbbbe3c332835c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER } }
I20260809 06:39:02.426087 6684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c9821b977247b183bbbbe3c332835c. Latest consensus state: current_term: 1 leader_uuid: "26c9821b977247b183bbbbe3c332835c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER } }
I20260809 06:39:02.426496 6684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:02.426501 6683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:02.434576 6687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:02.440539 6687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:02.441366 6707 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 06:39:02.450124 6687 catalog_manager.cc:1347] Generated new cluster ID: 55c47a86a2214103bdf596e5a51f3ed0
I20260809 06:39:02.450255 6687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:02.462370 6687 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:02.464223 6687 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:02.487685 6687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c9821b977247b183bbbbe3c332835c: Generated new TSK 0
I20260809 06:39:02.488781 6687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:02.498477 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:34323
--builtin_ntp_servers=127.6.111.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:02.852921 6710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:02.853235 6710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:02.853432 6710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:02.861078 6710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:02.861331 6710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:02.870802 6710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:34323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:02.872251 6710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:02.874207 6710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:02.888548 6717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:02.888648 6720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:02.888612 6718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:02.889636 6710 server_base.cc:1034] running on GCE node
I20260809 06:39:02.890275 6710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:02.891316 6710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:02.892601 6710 hybrid_clock.cc:648] HybridClock initialized: now 1786257542892503 us; error 69 us; skew 500 ppm
I20260809 06:39:02.892936 6710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:02.895299 6710 webserver.cc:460] Webserver started at http://127.6.111.1:35989/ using document root <none> and password file <none>
I20260809 06:39:02.896013 6710 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:02.896166 6710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:02.896485 6710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:02.899222 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "83a0435c0da74cd78cab122d7cc5ce20"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.900025 6710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "83a0435c0da74cd78cab122d7cc5ce20"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.905443 6710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 06:39:02.908998 6726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:02.910370 6710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 06:39:02.910561 6710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "83a0435c0da74cd78cab122d7cc5ce20"
format_stamp: "Formatted at 2026-08-09 06:39:02 on dist-test-slave-bw88"
I20260809 06:39:02.910774 6710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:02.922575 6710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:02.923363 6710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:02.923676 6710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:02.924680 6710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:02.926344 6710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:02.926451 6710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:02.926608 6710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:02.926682 6710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:02.977120 6710 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:35865
I20260809 06:39:02.977155 6868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:35865 every 8 connection(s)
I20260809 06:39:02.978602 6710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:02.983428 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 6710
I20260809 06:39:02.983698 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:02.996392 6869 heartbeater.cc:347] Connected to a master server at 127.6.111.62:34323
I20260809 06:39:02.996760 6869 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:02.997815 6869 heartbeater.cc:511] Master 127.6.111.62:34323 requested a full tablet report, sending...
I20260809 06:39:03.000382 6636 ts_manager.cc:194] Registered new tserver with Master: 83a0435c0da74cd78cab122d7cc5ce20 (127.6.111.1:35865)
I20260809 06:39:03.001855 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:03.002377 6636 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:48403
I20260809 06:39:03.006026 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
E20260809 06:39:03.994863 6636 master_service.cc:308] Failed adding master 127.6.111.61:41507. Invalid argument: 'last_known_addr' field in single master Raft configuration not set. Please restart master with --master_addresses flag set to the single master which will populate the 'last_known_addr' field.
I20260809 06:39:04.006469 6869 heartbeater.cc:503] Master 127.6.111.62:34323 was elected leader, sending a full tablet report...
I20260809 06:39:04.112020 6588 dynamic_multi_master-test.cc:554] Add master stderr: W20260809 06:39:03.402711 6873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:03.403053 6873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:03.403173 6873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:03.411453 6873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:03.411574 6873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:03.411638 6873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:03.411765 6873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:39:03.411855 6873 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20260809 06:39:03.774467 6885 logging.cc:291] disabling asynchronous logging adversely affects Kudu's performance
W20260809 06:39:03.781714 6885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:03.781840 6885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:03.781905 6885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:03.788599 6885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:03.788671 6885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:03.788724 6885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:03.788765 6885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:03.797358 6885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:34323,127.6.111.61:41507
--master_auto_join_cluster=false
--master_address_add_new_master=127.6.111.61:41507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:41507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_async=false
--log_filename=kudu
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:03.797622 6885 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:34323,127.6.111.61:41507'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:03.798672 6885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:03.800452 6885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:03.812107 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:03.812286 6889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:03.812436 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:03.813355 6885 server_base.cc:1034] running on GCE node
I20260809 06:39:03.814070 6885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:03.815456 6885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:03.816715 6885 hybrid_clock.cc:648] HybridClock initialized: now 1786257543816631 us; error 54 us; skew 500 ppm
I20260809 06:39:03.817056 6885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:03.819344 6885 webserver.cc:460] Webserver started at http://127.6.111.61:8051/ using document root <none> and password file <none>
I20260809 06:39:03.820011 6885 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:03.820175 6885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:03.820472 6885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:03.823176 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "9bf6c130afde45e285707c4e861c00f3"
format_stamp: "Formatted at 2026-08-09 06:39:03 on dist-test-slave-bw88"
I20260809 06:39:03.823928 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "9bf6c130afde45e285707c4e861c00f3"
format_stamp: "Formatted at 2026-08-09 06:39:03 on dist-test-slave-bw88"
I20260809 06:39:03.828981 6885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 06:39:03.832468 6899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:03.833762 6885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 06:39:03.833961 6885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "9bf6c130afde45e285707c4e861c00f3"
format_stamp: "Formatted at 2026-08-09 06:39:03 on dist-test-slave-bw88"
I20260809 06:39:03.834201 6885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:03.874843 6885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:03.875717 6885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:03.876013 6885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:03.895658 6885 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:41507
I20260809 06:39:03.895697 6963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:41507 every 8 connection(s)
I20260809 06:39:03.896993 6885 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:03.900898 6964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:03.907055 6964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 34323 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:03.921888 6964 sys_catalog.cc:422] member_type: NON_VOTER last_known_addr { host: "127.6.111.61" port: 41507 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:03.934648 6964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3: Bootstrap starting.
I20260809 06:39:03.937889 6964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:03.938964 6964 log.cc:824] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:03.941300 6964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3: No bootstrap required, opened a new log
I20260809 06:39:03.948334 6964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [term 0 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 34323 } } peers { permanent_uuid: "9bf6c130afde45e285707c4e861c00f3" member_type: NON_VOTER last_known_addr { host: "127.6.111.61" port: 41507 } }
I20260809 06:39:03.948683 6964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [term 0 LEARNER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:03.948796 6964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [term 0 LEARNER]: Becoming Follower/Learner. State: Replica: 9bf6c130afde45e285707c4e861c00f3, State: Initialized, Role: LEARNER
I20260809 06:39:03.949226 6964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 34323 } } peers { permanent_uuid: "9bf6c130afde45e285707c4e861c00f3" member_type: NON_VOTER last_known_addr { host: "127.6.111.61" port: 41507 } }
I20260809 06:39:03.950045 6970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c9821b977247b183bbbbe3c332835c" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 34323 } } peers { permanent_uuid: "9bf6c130afde45e285707c4e861c00f3" member_type: NON_VOTER last_known_addr { host: "127.6.111.61" port: 41507 } } }
I20260809 06:39:03.950364 6970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:39:03.951076 6964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:03.959331 6983 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bf6c130afde45e285707c4e861c00f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:03.959450 6983 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Remote error: Invalid argument: 'last_known_addr' field in single master Raft configuration not set. Please restart master with --master_addresses flag set to the single master which will populate the 'last_known_addr' field.
I20260809 06:39:04.117412 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 6710
I20260809 06:39:04.126940 6864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:04.212325 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 6710
I20260809 06:39:04.238940 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 6597
I20260809 06:39:04.240238 6670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:04.341377 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 6597
2026-08-09T06:39:04Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestAddMasterWithNoLastKnownAddr (2493 ms)
[ RUN ] DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname
2026-08-09T06:39:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:04Z Disabled control of system clock
I20260809 06:39:04.416146 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:32995
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:32995,127.6.111.61:43091
--master_auto_join_cluster=false with env {}
W20260809 06:39:04.755314 7002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:04.755615 7002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:04.755699 7002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:04.762701 7002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:04.762795 7002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:04.762838 7002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:04.762895 7002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:04.771580 7002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:32995,127.6.111.61:43091
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:32995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:04.771891 7002 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:32995,127.6.111.61:43091'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:04.772895 7002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:04.774710 7002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:04.786559 7007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:04.786621 7010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:04.786559 7008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:04.787879 7002 server_base.cc:1034] running on GCE node
I20260809 06:39:04.788759 7002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:04.790186 7002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:04.791433 7002 hybrid_clock.cc:648] HybridClock initialized: now 1786257544791363 us; error 49 us; skew 500 ppm
I20260809 06:39:04.791774 7002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:04.794103 7002 webserver.cc:460] Webserver started at http://127.6.111.62:42699/ using document root <none> and password file <none>
I20260809 06:39:04.794804 7002 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:04.794937 7002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:04.795279 7002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:04.799021 7002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "94e67a4010764347bef24700618c20a9"
format_stamp: "Formatted at 2026-08-09 06:39:04 on dist-test-slave-bw88"
I20260809 06:39:04.802757 7002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "94e67a4010764347bef24700618c20a9"
format_stamp: "Formatted at 2026-08-09 06:39:04 on dist-test-slave-bw88"
I20260809 06:39:04.808209 7002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:04.811633 7019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:04.812920 7002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:39:04.813118 7002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "94e67a4010764347bef24700618c20a9"
format_stamp: "Formatted at 2026-08-09 06:39:04 on dist-test-slave-bw88"
I20260809 06:39:04.813381 7002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:04.866287 7002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:04.867223 7002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:04.867533 7002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:04.886912 7002 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:32995
I20260809 06:39:04.886938 7087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:32995 every 8 connection(s)
I20260809 06:39:04.888285 7002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:04.892194 7088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:04.896338 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7002
I20260809 06:39:04.896551 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:04.898288 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:43091
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:32995,127.6.111.61:43091
--master_auto_join_cluster=false with env {}
I20260809 06:39:04.899904 7088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:04.920758 7088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:39:04.922124 7022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
W20260809 06:39:04.923781 7088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:43091: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
I20260809 06:39:04.973227 7088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } attempt: 1
W20260809 06:39:04.975448 7088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:43091: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
I20260809 06:39:05.043776 7088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } attempt: 2
W20260809 06:39:05.046010 7088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:43091: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
I20260809 06:39:05.137327 7088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } attempt: 3
W20260809 06:39:05.139357 7088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:43091: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
W20260809 06:39:05.239209 7093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:05.239499 7093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:05.239583 7093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:05.246536 7093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:05.246637 7093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:05.246681 7093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:05.246738 7093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:05.255520 7093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:32995,127.6.111.61:43091
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:43091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:05.255831 7093 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:32995,127.6.111.61:43091'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:05.256847 7093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:05.258569 7093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:05.270750 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:05.270871 7101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:05.270944 7100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:05.272070 7093 server_base.cc:1034] running on GCE node
I20260809 06:39:05.272908 7093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:05.274493 7093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:05.275728 7093 hybrid_clock.cc:648] HybridClock initialized: now 1786257545275696 us; error 58 us; skew 500 ppm
I20260809 06:39:05.276086 7093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:05.278479 7093 webserver.cc:460] Webserver started at http://127.6.111.61:34409/ using document root <none> and password file <none>
I20260809 06:39:05.279217 7093 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:05.279371 7093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:05.279672 7093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:05.282634 7088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } attempt: 4
I20260809 06:39:05.282524 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "2f01a325307a4c2e9a3e4e306779dce9"
format_stamp: "Formatted at 2026-08-09 06:39:05 on dist-test-slave-bw88"
I20260809 06:39:05.283501 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "2f01a325307a4c2e9a3e4e306779dce9"
format_stamp: "Formatted at 2026-08-09 06:39:05 on dist-test-slave-bw88"
W20260809 06:39:05.284691 7088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:43091: Network error: Client connection negotiation failed: client connection to 127.6.111.61:43091: connect: Connection refused (error 111)
I20260809 06:39:05.289237 7093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 06:39:05.292721 7111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:05.294093 7093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:05.294284 7093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "2f01a325307a4c2e9a3e4e306779dce9"
format_stamp: "Formatted at 2026-08-09 06:39:05 on dist-test-slave-bw88"
I20260809 06:39:05.294556 7093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:05.319712 7093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:05.320683 7093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:05.320989 7093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:05.339406 7093 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:43091
I20260809 06:39:05.339442 7177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:43091 every 8 connection(s)
I20260809 06:39:05.340806 7093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:05.344565 7178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:05.350847 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7093
I20260809 06:39:05.351001 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:05.350663 7178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:05.373104 7178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:05.382916 7178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: Bootstrap starting.
I20260809 06:39:05.386260 7178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:05.387279 7178 log.cc:824] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:05.389909 7178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: No bootstrap required, opened a new log
I20260809 06:39:05.396925 7178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.397257 7178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:05.397385 7178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f01a325307a4c2e9a3e4e306779dce9, State: Initialized, Role: FOLLOWER
I20260809 06:39:05.397923 7178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.398840 7185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.399176 7185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:05.399735 7178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:05.407896 7207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:05.408059 7207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 06:39:05.573298 7118 tablet.cc:2369] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:05.583930 7088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } attempt: 5
I20260809 06:39:05.594194 7088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: Bootstrap starting.
I20260809 06:39:05.597589 7088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:05.598747 7088 log.cc:824] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:05.601612 7088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: No bootstrap required, opened a new log
I20260809 06:39:05.609612 7088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.609918 7088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:05.610021 7088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e67a4010764347bef24700618c20a9, State: Initialized, Role: FOLLOWER
I20260809 06:39:05.610683 7088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.611650 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.611996 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:05.612866 7088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:05.620095 7027 tablet.cc:2369] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:39:05.623020 7233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:05.623158 7233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:05.694002 7185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:05.694408 7185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.696096 7185 leader_election.cc:288] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94e67a4010764347bef24700618c20a9 (127.6.111.62:32995)
I20260809 06:39:05.697214 7053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f01a325307a4c2e9a3e4e306779dce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e67a4010764347bef24700618c20a9" is_pre_election: true
I20260809 06:39:05.698014 7053 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f01a325307a4c2e9a3e4e306779dce9 in term 0.
I20260809 06:39:05.698793 7113 leader_election.cc:302] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2f01a325307a4c2e9a3e4e306779dce9, 94e67a4010764347bef24700618c20a9; no voters:
I20260809 06:39:05.699434 7185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:39:05.699606 7185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:05.699709 7185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:05.703125 7185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.703797 7185 leader_election.cc:288] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [CANDIDATE]: Term 1 election: Requested vote from peers 94e67a4010764347bef24700618c20a9 (127.6.111.62:32995)
I20260809 06:39:05.704380 7053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f01a325307a4c2e9a3e4e306779dce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e67a4010764347bef24700618c20a9"
I20260809 06:39:05.704644 7053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:05.708212 7053 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f01a325307a4c2e9a3e4e306779dce9 in term 1.
I20260809 06:39:05.708693 7113 leader_election.cc:302] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 2f01a325307a4c2e9a3e4e306779dce9, 94e67a4010764347bef24700618c20a9; no voters:
I20260809 06:39:05.709121 7185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:05.713008 7185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [term 1 LEADER]: Becoming Leader. State: Replica: 2f01a325307a4c2e9a3e4e306779dce9, State: Running, Role: LEADER
I20260809 06:39:05.713675 7185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } }
I20260809 06:39:05.722705 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f01a325307a4c2e9a3e4e306779dce9. Latest consensus state: current_term: 1 leader_uuid: "2f01a325307a4c2e9a3e4e306779dce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.723109 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:05.724215 7239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:05.726691 7239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:05.735713 7053 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [term 1 FOLLOWER]: Refusing update from remote peer 2f01a325307a4c2e9a3e4e306779dce9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:39:05.737422 7235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:05.745504 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f01a325307a4c2e9a3e4e306779dce9. Latest consensus state: current_term: 1 leader_uuid: "2f01a325307a4c2e9a3e4e306779dce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.745754 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:05.757822 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f01a325307a4c2e9a3e4e306779dce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.758103 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:05.760458 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f01a325307a4c2e9a3e4e306779dce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e67a4010764347bef24700618c20a9" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 32995 } } peers { permanent_uuid: "2f01a325307a4c2e9a3e4e306779dce9" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 43091 } } }
I20260809 06:39:05.760743 7239 catalog_manager.cc:1347] Generated new cluster ID: e43abafba96e486fa37095202c814e9b
I20260809 06:39:05.760738 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:05.760844 7239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:05.796069 7239 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:05.798131 7239 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:05.813081 7239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f01a325307a4c2e9a3e4e306779dce9: Generated new TSK 0
I20260809 06:39:05.814114 7239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:05.831616 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:32995,127.6.111.61:43091
--builtin_ntp_servers=127.6.111.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:06.170936 7245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:06.171356 7245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:06.171519 7245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:06.178735 7245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:06.178977 7245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:06.188261 7245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:32995,127.6.111.61:43091
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:06.189697 7245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:06.191660 7245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:06.205262 7252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:06.205299 7254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:06.205215 7256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:06.206095 7245 server_base.cc:1034] running on GCE node
I20260809 06:39:06.206717 7245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:06.207574 7245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:06.208788 7245 hybrid_clock.cc:648] HybridClock initialized: now 1786257546208705 us; error 53 us; skew 500 ppm
I20260809 06:39:06.209045 7245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:06.211428 7245 webserver.cc:460] Webserver started at http://127.6.111.1:33767/ using document root <none> and password file <none>
I20260809 06:39:06.212069 7245 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:06.212167 7245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:06.212450 7245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:06.215204 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "2bf1dcac7a49482281e5adeea275e922"
format_stamp: "Formatted at 2026-08-09 06:39:06 on dist-test-slave-bw88"
I20260809 06:39:06.215982 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "2bf1dcac7a49482281e5adeea275e922"
format_stamp: "Formatted at 2026-08-09 06:39:06 on dist-test-slave-bw88"
I20260809 06:39:06.221297 7245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:39:06.224576 7262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:06.225886 7245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:06.226019 7245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "2bf1dcac7a49482281e5adeea275e922"
format_stamp: "Formatted at 2026-08-09 06:39:06 on dist-test-slave-bw88"
I20260809 06:39:06.226166 7245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:06.241236 7245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:06.242048 7245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:06.242298 7245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:06.243139 7245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:06.244544 7245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:06.244626 7245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:06.244726 7245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:06.244771 7245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:06.276955 7245 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:34059
I20260809 06:39:06.276996 7408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:34059 every 8 connection(s)
I20260809 06:39:06.278348 7245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:06.283317 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7245
I20260809 06:39:06.283586 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:06.310925 7409 heartbeater.cc:347] Connected to a master server at 127.6.111.61:43091
I20260809 06:39:06.310909 7410 heartbeater.cc:347] Connected to a master server at 127.6.111.62:32995
I20260809 06:39:06.311333 7410 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:06.311334 7409 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:06.312446 7409 heartbeater.cc:511] Master 127.6.111.61:43091 requested a full tablet report, sending...
I20260809 06:39:06.312513 7410 heartbeater.cc:511] Master 127.6.111.62:32995 requested a full tablet report, sending...
I20260809 06:39:06.315001 7135 ts_manager.cc:194] Registered new tserver with Master: 2bf1dcac7a49482281e5adeea275e922 (127.6.111.1:34059)
I20260809 06:39:06.315191 7040 ts_manager.cc:194] Registered new tserver with Master: 2bf1dcac7a49482281e5adeea275e922 (127.6.111.1:34059)
I20260809 06:39:06.317111 7135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:34171
I20260809 06:39:06.318228 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:06.323122 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:06.625746 7233 catalog_manager.cc:1259] Loaded cluster ID: e43abafba96e486fa37095202c814e9b
I20260809 06:39:06.625867 7233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: loading cluster ID for follower catalog manager: success
I20260809 06:39:06.629863 7233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: acquiring CA information for follower catalog manager: success
I20260809 06:39:06.632344 7233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 94e67a4010764347bef24700618c20a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
E20260809 06:39:07.129767 7135 master_service.cc:344] Failed removing master non-existent-path.local:7051. Not found: Master non-existent-path.local:7051 not found
I20260809 06:39:07.222147 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7245
I20260809 06:39:07.231289 7404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:07.313748 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7245
I20260809 06:39:07.340641 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7002
I20260809 06:39:07.342024 7079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:07.480850 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7002
I20260809 06:39:07.511013 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7093
I20260809 06:39:07.512437 7172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:07.618822 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7093
2026-08-09T06:39:07Z chronyd exiting
[ OK ] DynamicMultiMasterTest.TestRemoveMasterMissingAndIncorrectHostname (3277 ms)
[----------] 2 tests from DynamicMultiMasterTest (5770 ms total)
[----------] 1 test from AutoAddMasterTest
[ RUN ] AutoAddMasterTest.TestFailTabletCopy
2026-08-09T06:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:07Z Disabled control of system clock
I20260809 06:39:07.698565 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:43033
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967 with env {}
W20260809 06:39:08.040274 7449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:08.040567 7449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:08.040645 7449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:08.047909 7449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:08.048013 7449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:08.048055 7449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:08.048105 7449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:08.057034 7449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:08.057399 7449 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:43033,127.6.111.61:33967'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:08.058507 7449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:08.060242 7449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:08.072541 7455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:08.072490 7456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:08.072554 7458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:08.073798 7449 server_base.cc:1034] running on GCE node
I20260809 06:39:08.074581 7449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:08.076002 7449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:08.077239 7449 hybrid_clock.cc:648] HybridClock initialized: now 1786257548077183 us; error 51 us; skew 500 ppm
I20260809 06:39:08.077618 7449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:08.079952 7449 webserver.cc:460] Webserver started at http://127.6.111.62:42429/ using document root <none> and password file <none>
I20260809 06:39:08.080696 7449 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:08.080842 7449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:08.081218 7449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:08.084041 7449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "e38e1cb910d54113b784b58b509ef675"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.084924 7449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "e38e1cb910d54113b784b58b509ef675"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.090394 7449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 06:39:08.093981 7466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:08.095335 7449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:39:08.095531 7449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "e38e1cb910d54113b784b58b509ef675"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.095796 7449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:08.129914 7449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:08.130897 7449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:08.131222 7449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:08.150688 7449 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:43033
I20260809 06:39:08.150717 7533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:43033 every 8 connection(s)
I20260809 06:39:08.152112 7449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:08.156036 7535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:08.158150 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7449
I20260809 06:39:08.158344 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:08.159550 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:33967
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967 with env {}
I20260809 06:39:08.162488 7535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:08.181831 7535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:39:08.183094 7468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
W20260809 06:39:08.184715 7535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:33967: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
I20260809 06:39:08.234081 7535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } attempt: 1
W20260809 06:39:08.236063 7535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:33967: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
I20260809 06:39:08.304383 7535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } attempt: 2
W20260809 06:39:08.306543 7535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:33967: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
I20260809 06:39:08.397826 7535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } attempt: 3
W20260809 06:39:08.399966 7535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:33967: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
W20260809 06:39:08.496778 7538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:08.497033 7538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:08.497107 7538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:08.504581 7538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:08.504685 7538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:08.504736 7538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:08.504792 7538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:08.513650 7538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:08.513935 7538 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:43033,127.6.111.61:33967'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:08.515213 7538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:08.517776 7538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:08.529850 7549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:08.529846 7552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:08.530706 7555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:08.530689 7538 server_base.cc:1034] running on GCE node
I20260809 06:39:08.531497 7538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:08.532722 7538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:08.533921 7538 hybrid_clock.cc:648] HybridClock initialized: now 1786257548533877 us; error 42 us; skew 500 ppm
I20260809 06:39:08.534271 7538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:08.536571 7538 webserver.cc:460] Webserver started at http://127.6.111.61:36889/ using document root <none> and password file <none>
I20260809 06:39:08.537228 7538 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:08.537343 7538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:08.537720 7538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:08.540279 7538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "3b5c031449f24ef3aee05c867d282717"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.541061 7538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "3b5c031449f24ef3aee05c867d282717"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.543243 7535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } attempt: 4
W20260809 06:39:08.545277 7535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:33967: Network error: Client connection negotiation failed: client connection to 127.6.111.61:33967: connect: Connection refused (error 111)
I20260809 06:39:08.546429 7538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 06:39:08.549798 7564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:08.551151 7538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 06:39:08.551342 7538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "3b5c031449f24ef3aee05c867d282717"
format_stamp: "Formatted at 2026-08-09 06:39:08 on dist-test-slave-bw88"
I20260809 06:39:08.551569 7538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:08.574473 7538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:08.575345 7538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:08.575668 7538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:08.595160 7538 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:33967
I20260809 06:39:08.595182 7625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:33967 every 8 connection(s)
I20260809 06:39:08.596590 7538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:08.600836 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:08.602409 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7538
I20260809 06:39:08.602697 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:08.609570 7626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:08.622767 7626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:08.633069 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Bootstrap starting.
I20260809 06:39:08.636751 7626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:08.637915 7626 log.cc:824] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:08.640622 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: No bootstrap required, opened a new log
I20260809 06:39:08.648765 7626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.649109 7626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:08.649211 7626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5c031449f24ef3aee05c867d282717, State: Initialized, Role: FOLLOWER
I20260809 06:39:08.649883 7626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.650727 7632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:08.651018 7632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:08.651806 7626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:08.661734 7648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:08.661859 7648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:08.659227 7538 master_runner.cc:186] Error getting master registration for 127.6.111.62:43033: OK, instance_id { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" instance_seqno: 1786257548145206 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:39:08.668109 7538 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 06:39:08.827950 7571 tablet.cc:2369] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:08.844540 7535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } attempt: 5
I20260809 06:39:08.857245 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: Bootstrap starting.
I20260809 06:39:08.861868 7535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:08.863225 7535 log.cc:824] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:08.866241 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: No bootstrap required, opened a new log
I20260809 06:39:08.877454 7535 raft_consensus.cc:348] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.877890 7535 raft_consensus.cc:374] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:08.878000 7535 raft_consensus.cc:729] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38e1cb910d54113b784b58b509ef675, State: Initialized, Role: FOLLOWER
I20260809 06:39:08.878688 7535 consensus_queue.cc:260] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.879637 7652 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:08.879959 7652 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:08.881050 7535 sys_catalog.cc:564] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:08.883822 7478 tablet.cc:2369] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:39:08.890909 7670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:08.891095 7670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:08.893246 7449 master_runner.cc:263] No leader master found from master e38e1cb910d54113b784b58b509ef675
I20260809 06:39:08.893388 7449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:39:08.942231 7632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:08.942624 7632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.944314 7632 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e38e1cb910d54113b784b58b509ef675 (127.6.111.62:43033)
I20260809 06:39:08.945315 7507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b5c031449f24ef3aee05c867d282717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38e1cb910d54113b784b58b509ef675" is_pre_election: true
I20260809 06:39:08.946097 7507 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5c031449f24ef3aee05c867d282717 in term 0.
I20260809 06:39:08.946808 7569 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3b5c031449f24ef3aee05c867d282717, e38e1cb910d54113b784b58b509ef675; no voters:
I20260809 06:39:08.947230 7632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:39:08.947402 7632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:08.947507 7632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:08.950829 7632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.951392 7632 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [CANDIDATE]: Term 1 election: Requested vote from peers e38e1cb910d54113b784b58b509ef675 (127.6.111.62:43033)
I20260809 06:39:08.952001 7507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b5c031449f24ef3aee05c867d282717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38e1cb910d54113b784b58b509ef675"
I20260809 06:39:08.952232 7507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:08.955746 7507 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5c031449f24ef3aee05c867d282717 in term 1.
I20260809 06:39:08.957159 7569 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3b5c031449f24ef3aee05c867d282717, e38e1cb910d54113b784b58b509ef675; no voters:
I20260809 06:39:08.958178 7632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:08.958824 7632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5c031449f24ef3aee05c867d282717, State: Running, Role: LEADER
I20260809 06:39:08.959533 7632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:08.963872 7672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5c031449f24ef3aee05c867d282717. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:08.964278 7672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:08.965178 7674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:08.966974 7674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:08.974309 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:39:08.975235 7672 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:08.985901 7652 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5c031449f24ef3aee05c867d282717. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:08.986176 7652 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:08.988395 7676 mvcc.cc:205] Tried to move back new op lower bound from 7316510920589725696 to 7316510920542806016. Current Snapshot: MvccSnapshot[applied={T|T < 7316510920589725696}]
I20260809 06:39:09.003618 7672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:09.004025 7672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:09.004947 7652 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:09.005190 7652 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:09.007167 7674 catalog_manager.cc:1347] Generated new cluster ID: 9fb63a6e02234b7abb12f8e757bb6a25
I20260809 06:39:09.007275 7674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:09.040113 7674 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:09.042248 7674 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:09.058336 7674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Generated new TSK 0
I20260809 06:39:09.059119 7674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:09.073762 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:43033,127.6.111.61:33967
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:09.417183 7681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:09.417554 7681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:09.417645 7681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:09.424837 7681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:09.424989 7681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:09.433998 7681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:43033,127.6.111.61:33967
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:09.435331 7681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:09.437520 7681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:09.450994 7687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:09.451050 7688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:09.451071 7691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:09.451891 7681 server_base.cc:1034] running on GCE node
I20260809 06:39:09.452499 7681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:09.453506 7681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:09.454761 7681 hybrid_clock.cc:648] HybridClock initialized: now 1786257549454667 us; error 54 us; skew 500 ppm
I20260809 06:39:09.455086 7681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:09.457504 7681 webserver.cc:460] Webserver started at http://127.6.111.1:35343/ using document root <none> and password file <none>
I20260809 06:39:09.458182 7681 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:09.458323 7681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:09.458639 7681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:09.461265 7681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "d496a732f4064e9a8d21acba3a5b3971"
format_stamp: "Formatted at 2026-08-09 06:39:09 on dist-test-slave-bw88"
I20260809 06:39:09.462100 7681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "d496a732f4064e9a8d21acba3a5b3971"
format_stamp: "Formatted at 2026-08-09 06:39:09 on dist-test-slave-bw88"
I20260809 06:39:09.467222 7681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 06:39:09.470643 7697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:09.471976 7681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:39:09.472162 7681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "d496a732f4064e9a8d21acba3a5b3971"
format_stamp: "Formatted at 2026-08-09 06:39:09 on dist-test-slave-bw88"
I20260809 06:39:09.472360 7681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:09.483882 7681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:09.484692 7681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:09.485004 7681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:09.486019 7681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:09.487448 7681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:09.487545 7681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:09.487635 7681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:09.487680 7681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:09.522923 7681 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:43105
I20260809 06:39:09.522974 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:43105 every 8 connection(s)
I20260809 06:39:09.524215 7681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:09.525805 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7681
I20260809 06:39:09.526039 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:09.539660 7841 heartbeater.cc:347] Connected to a master server at 127.6.111.61:33967
I20260809 06:39:09.539685 7843 heartbeater.cc:347] Connected to a master server at 127.6.111.62:43033
I20260809 06:39:09.540000 7841 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:09.540004 7843 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:09.540835 7841 heartbeater.cc:511] Master 127.6.111.61:33967 requested a full tablet report, sending...
I20260809 06:39:09.540855 7843 heartbeater.cc:511] Master 127.6.111.62:43033 requested a full tablet report, sending...
I20260809 06:39:09.542963 7589 ts_manager.cc:194] Registered new tserver with Master: d496a732f4064e9a8d21acba3a5b3971 (127.6.111.1:43105)
I20260809 06:39:09.543320 7493 ts_manager.cc:194] Registered new tserver with Master: d496a732f4064e9a8d21acba3a5b3971 (127.6.111.1:43105)
I20260809 06:39:09.544852 7589 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:51303
I20260809 06:39:09.546692 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:09.551633 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:09.551961 6588 test_util.cc:268] Using random seed: 2125982003
I20260809 06:39:09.572819 7589 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58394:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:09.599192 7751 tablet_service.cc:1467] Processing CreateTablet for tablet 48ce55ef3b644f0bb384777b94214bdc (DEFAULT_TABLE table=test-workload [id=2f0978615efc4192ab505b65958f78e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:09.600530 7751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ce55ef3b644f0bb384777b94214bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:09.611378 7870 tablet_bootstrap.cc:492] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: Bootstrap starting.
I20260809 06:39:09.614598 7870 tablet_bootstrap.cc:654] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:09.615557 7870 log.cc:824] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:09.617910 7870 tablet_bootstrap.cc:492] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: No bootstrap required, opened a new log
I20260809 06:39:09.618145 7870 ts_tablet_manager.cc:1397] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 06:39:09.625270 7870 raft_consensus.cc:348] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d496a732f4064e9a8d21acba3a5b3971" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43105 } }
I20260809 06:39:09.625672 7870 raft_consensus.cc:374] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:09.625754 7870 raft_consensus.cc:729] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d496a732f4064e9a8d21acba3a5b3971, State: Initialized, Role: FOLLOWER
I20260809 06:39:09.626267 7870 consensus_queue.cc:260] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d496a732f4064e9a8d21acba3a5b3971" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43105 } }
I20260809 06:39:09.626470 7870 raft_consensus.cc:388] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:09.626571 7870 raft_consensus.cc:482] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:09.626695 7870 raft_consensus.cc:3037] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:09.628984 7870 raft_consensus.cc:504] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d496a732f4064e9a8d21acba3a5b3971" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43105 } }
I20260809 06:39:09.629642 7870 leader_election.cc:302] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d496a732f4064e9a8d21acba3a5b3971; no voters:
I20260809 06:39:09.630251 7870 leader_election.cc:288] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:09.630409 7872 raft_consensus.cc:2781] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:09.631117 7872 raft_consensus.cc:686] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [term 1 LEADER]: Becoming Leader. State: Replica: d496a732f4064e9a8d21acba3a5b3971, State: Running, Role: LEADER
I20260809 06:39:09.631395 7841 heartbeater.cc:503] Master 127.6.111.61:33967 was elected leader, sending a full tablet report...
I20260809 06:39:09.631759 7872 consensus_queue.cc:237] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d496a732f4064e9a8d21acba3a5b3971" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43105 } }
I20260809 06:39:09.632213 7870 ts_tablet_manager.cc:1428] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 06:39:09.638015 7589 catalog_manager.cc:5495] T 48ce55ef3b644f0bb384777b94214bdc P d496a732f4064e9a8d21acba3a5b3971 reported cstate change: term changed from 0 to 1, leader changed from <none> to d496a732f4064e9a8d21acba3a5b3971 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "d496a732f4064e9a8d21acba3a5b3971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d496a732f4064e9a8d21acba3a5b3971" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43105 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:09.656157 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--webserver_interface=127.6.111.60
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--tablet_copy_fault_crash_during_download_wal=1 with env {}
I20260809 06:39:09.893163 7670 catalog_manager.cc:1259] Loaded cluster ID: 9fb63a6e02234b7abb12f8e757bb6a25
I20260809 06:39:09.893293 7670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: loading cluster ID for follower catalog manager: success
I20260809 06:39:09.895895 7670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: acquiring CA information for follower catalog manager: success
I20260809 06:39:09.897706 7670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 06:39:10.001436 7881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:10.001833 7881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:10.001996 7881 flags.cc:425] Enabled unsafe flag: --tablet_copy_fault_crash_during_download_wal=1
W20260809 06:39:10.002060 7881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:10.009143 7881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:10.009222 7881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:10.009308 7881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:10.009346 7881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:10.018340 7881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.111.60
--webserver_port=0
--tablet_copy_fault_crash_during_download_wal=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:10.019690 7881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:10.021632 7881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:10.033701 7887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:10.033768 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:10.033736 7891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:10.034593 7881 server_base.cc:1034] running on GCE node
I20260809 06:39:10.035274 7881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:10.036587 7881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:10.037782 7881 hybrid_clock.cc:648] HybridClock initialized: now 1786257550037730 us; error 41 us; skew 500 ppm
I20260809 06:39:10.038128 7881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:10.040445 7881 webserver.cc:460] Webserver started at http://127.6.111.60:42453/ using document root <none> and password file <none>
I20260809 06:39:10.041157 7881 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:10.041275 7881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:10.041667 7881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:10.044323 7881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/instance:
uuid: "0a127e3b50244511b0692b17845b3876"
format_stamp: "Formatted at 2026-08-09 06:39:10 on dist-test-slave-bw88"
I20260809 06:39:10.045465 7881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal/instance:
uuid: "0a127e3b50244511b0692b17845b3876"
format_stamp: "Formatted at 2026-08-09 06:39:10 on dist-test-slave-bw88"
I20260809 06:39:10.050629 7881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 06:39:10.054301 7898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:10.055822 7881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:39:10.056012 7881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "0a127e3b50244511b0692b17845b3876"
format_stamp: "Formatted at 2026-08-09 06:39:10 on dist-test-slave-bw88"
I20260809 06:39:10.056284 7881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:10.085263 7881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:10.086225 7881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:10.086582 7881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:10.106877 7881 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:36913
I20260809 06:39:10.106901 7963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:36913 every 8 connection(s)
I20260809 06:39:10.108441 7881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:10.112340 7964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:10.118073 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 7881
I20260809 06:39:10.118857 7964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:10.133769 7964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:10.140480 7964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:10.155190 7964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Bootstrap starting.
I20260809 06:39:10.159255 7964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:10.160558 7964 log.cc:824] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:10.164355 7964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: No bootstrap required, opened a new log
I20260809 06:39:10.173954 7964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:10.174297 7964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:10.174386 7964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a127e3b50244511b0692b17845b3876, State: Initialized, Role: FOLLOWER
I20260809 06:39:10.174929 7964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:10.175894 7974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } } }
I20260809 06:39:10.176321 7974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:10.177167 7964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:10.186856 7992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:10.187017 7992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:10.191931 7881 master_runner.cc:428] Detected that this master 0a127e3b50244511b0692b17845b3876 is joining an existing cluster
I20260809 06:39:10.192071 7881 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.111.60:36913
I20260809 06:39:10.226473 7589 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.111.60:36913: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0a127e3b50244511b0692b17845b3876"
member_type: NON_VOTER
last_known_addr {
host: "127.6.111.60"
port: 36913
}
attrs {
promote: true
}
}
dest_uuid: "3b5c031449f24ef3aee05c867d282717"
cas_config_opid_index: -1
I20260809 06:39:10.227864 7589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } }
I20260809 06:39:10.230595 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 06:39:10.230767 7931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:10.231475 8004 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 06:39:10.234596 7931 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 06:39:10.234578 7864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER 0a127e3b50244511b0692b17845b3876 (127.6.111.60) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:10.237324 7864 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.111.60:36913
I20260809 06:39:10.237916 8004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:10.238312 8004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:10.238471 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:10.238914 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:10.238268 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER 0a127e3b50244511b0692b17845b3876 (127.6.111.60) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:10.243798 7864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:10.244553 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:10.244923 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:10.245559 7881 master.cc:537] Master@127.6.111.60:36913 shutting down...
W20260809 06:39:10.247828 7569 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 06:39:10.249433 7569 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 -> Peer 0a127e3b50244511b0692b17845b3876 (127.6.111.60:36913): Couldn't send request to peer 0a127e3b50244511b0692b17845b3876. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 06:39:10.275066 7881 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:39:10.275568 7881 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:39:10.275765 7881 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: stopping tablet replica
I20260809 06:39:10.282333 7881 master.cc:552] Master@127.6.111.60:36913 shutdown complete.
I20260809 06:39:10.282577 7881 master_runner.cc:305] Clearing existing system tablet
I20260809 06:39:10.283015 7881 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:10.283128 7881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:10.285630 7881 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 06:39:10.287106 8010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:10.287438 7881 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:10.287747 7881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 06:39:10.287828 7881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "0a127e3b50244511b0692b17845b3876"
format_stamp: "Formatted at 2026-08-09 06:39:10 on dist-test-slave-bw88"
I20260809 06:39:10.291185 7881 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:39:10.296044 7881 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 06:39:10.296231 7881 log.cc:1196] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Deleting WAL directory at /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 06:39:10.296657 7881 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Deleting consensus metadata
I20260809 06:39:10.296923 7881 master_runner.cc:315] Copying system tablet from 127.6.111.61:33967
I20260809 06:39:10.299438 7881 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: tablet copy: Beginning tablet copy session from remote peer at address 127.6.111.61:33967
I20260809 06:39:10.307184 7611 tablet_copy_service.cc:140] P 3b5c031449f24ef3aee05c867d282717: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0a127e3b50244511b0692b17845b3876 ({username='slave'} at 127.0.0.1:58426)
I20260809 06:39:10.307427 7611 tablet_copy_service.cc:161] P 3b5c031449f24ef3aee05c867d282717: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0a127e3b50244511b0692b17845b3876 at {username='slave'} at 127.0.0.1:58426: session id = 0a127e3b50244511b0692b17845b3876-00000000000000000000000000000000
I20260809 06:39:10.310365 7611 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 06:39:10.312135 7881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 06:39:10.316450 7881 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: tablet copy: Starting download of 0 data blocks...
I20260809 06:39:10.316732 7881 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: tablet copy: Starting download of 1 WAL segments...
W20260809 06:39:10.316989 8019 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 06:39:10.317055 8019 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260809 06:39:10.317857 7611 tablet_copy_service.cc:342] P 3b5c031449f24ef3aee05c867d282717: Request end of tablet copy session 0a127e3b50244511b0692b17845b3876-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:58426
I20260809 06:39:10.318041 7611 tablet_copy_service.cc:433] P 3b5c031449f24ef3aee05c867d282717: ending tablet copy session 0a127e3b50244511b0692b17845b3876-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0a127e3b50244511b0692b17845b3876
W20260809 06:39:10.319080 7881 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet 00000000000000000000000000000000 but one is already registered
I20260809 06:39:10.319283 7881 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 06:39:10.324064 7881 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20260809 06:39:10.324219 7881 log.cc:1196] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876: Deleting WAL directory at /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
IO error: Injected failure on downloading wal
I20260809 06:39:11.145113 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--webserver_interface=127.6.111.60
--webserver_port=42453
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--tablet_copy_fault_crash_during_download_wal=1
--tablet_copy_fault_crash_during_download_wal=0 with env {}
W20260809 06:39:11.485862 8028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:11.486176 8028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:11.486349 8028 flags.cc:425] Enabled unsafe flag: --tablet_copy_fault_crash_during_download_wal=0
W20260809 06:39:11.486403 8028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:11.493656 8028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:11.493763 8028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:11.493855 8028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:11.493923 8028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:11.503304 8028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.111.60
--webserver_port=42453
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:11.504628 8028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:11.506462 8028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:11.518249 8037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:11.518337 8038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:11.518306 8040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:11.519742 8028 server_base.cc:1034] running on GCE node
I20260809 06:39:11.520437 8028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:11.521799 8028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:11.523007 8028 hybrid_clock.cc:648] HybridClock initialized: now 1786257551522962 us; error 35 us; skew 500 ppm
I20260809 06:39:11.523360 8028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:11.525753 8028 webserver.cc:460] Webserver started at http://127.6.111.60:42453/ using document root <none> and password file <none>
I20260809 06:39:11.526429 8028 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:11.526566 8028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:11.532871 8028 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 06:39:11.535598 8047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:11.536880 8028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:39:11.537058 8028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "0a127e3b50244511b0692b17845b3876"
format_stamp: "Formatted at 2026-08-09 06:39:10 on dist-test-slave-bw88"
I20260809 06:39:11.538069 8028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:11.562623 8028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:11.563388 8028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:11.563705 8028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:11.582372 8028 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:36913
I20260809 06:39:11.582548 8113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:36913 every 8 connection(s)
I20260809 06:39:11.583940 8028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:11.588292 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8028
I20260809 06:39:11.590583 8116 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:11.594933 8116 tablet_replica.cc:330] stopping tablet replica
I20260809 06:39:11.595180 8116 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 1 LEARNER]: Raft consensus shutting down.
I20260809 06:39:11.595885 8116 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0a127e3b50244511b0692b17845b3876 [term 1 LEARNER]: Raft consensus is shut down!
E20260809 06:39:11.596576 8116 master.cc:392] Unable to init master catalog manager: Corruption: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Unable to locally bootstrap tablet 00000000000000000000000000000000: TabletMetadata bootstrap state is TABLET_DATA_TOMBSTONED
I20260809 06:39:11.596872 8028 master.cc:537] Master@127.6.111.60:36913 shutting down...
I20260809 06:39:11.619997 8028 master.cc:552] Master@127.6.111.60:36913 shutdown complete.
Corruption: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Unable to locally bootstrap tablet 00000000000000000000000000000000: TabletMetadata bootstrap state is TABLET_DATA_TOMBSTONED
I20260809 06:39:11.847016 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--webserver_interface=127.6.111.60
--webserver_port=42453
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--tablet_copy_fault_crash_during_download_wal=1
--tablet_copy_fault_crash_during_download_wal=0 with env {}
W20260809 06:39:12.178627 8119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:12.178913 8119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:12.179004 8119 flags.cc:425] Enabled unsafe flag: --tablet_copy_fault_crash_during_download_wal=0
W20260809 06:39:12.179041 8119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:12.186241 8119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:12.186319 8119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:12.186370 8119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:12.186414 8119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:12.194962 8119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.111.60
--webserver_port=42453
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:12.196156 8119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:12.198087 8119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:12.209707 8127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:12.209707 8125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:12.209952 8124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:12.210567 8119 server_base.cc:1034] running on GCE node
I20260809 06:39:12.211366 8119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:12.212648 8119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:12.213872 8119 hybrid_clock.cc:648] HybridClock initialized: now 1786257552213829 us; error 56 us; skew 500 ppm
I20260809 06:39:12.214236 8119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:12.216542 8119 webserver.cc:460] Webserver started at http://127.6.111.60:42453/ using document root <none> and password file <none>
I20260809 06:39:12.217245 8119 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:12.217422 8119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:12.217805 8119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:12.220368 8119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/instance:
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:12.221163 8119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal/instance:
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:12.226245 8119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:12.229518 8134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:12.230762 8119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:39:12.230948 8119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:12.231196 8119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:12.247390 8119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:12.248178 8119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:12.248451 8119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:12.267122 8119 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:36913
I20260809 06:39:12.267171 8204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:36913 every 8 connection(s)
I20260809 06:39:12.268436 8119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:12.272089 8205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:12.277966 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8119
I20260809 06:39:12.278702 8205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:12.292222 8205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:12.298592 8205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:12.310367 8205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap starting.
I20260809 06:39:12.313570 8205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:12.314600 8205 log.cc:824] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:12.316991 8205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: No bootstrap required, opened a new log
I20260809 06:39:12.326305 8205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:12.326607 8205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:12.326776 8205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ace6e0f1df74902b713466fa639b837, State: Initialized, Role: FOLLOWER
I20260809 06:39:12.327450 8205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:12.328439 8213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } } }
I20260809 06:39:12.328882 8213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 06:39:12.327550 8175 tablet_service.cc:374] Invalid argument: UpdateConsensus: Wrong destination UUID requested. Local UUID: 1ace6e0f1df74902b713466fa639b837. Requested UUID: 0a127e3b50244511b0692b17845b3876: from {username='slave'} at 127.0.0.1:48430: tablet_id: "00000000000000000000000000000000" caller_uuid: "3b5c031449f24ef3aee05c867d282717" caller_term: 1 preceding_id { term: 0 index: 0 } ops { id { term: 1 index: 1 } timestamp: 7316510920542806016 op_type: NO_OP noop_request { } } ops { id { term: 1 index: 2 } timestamp: 7316510920589725696 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\001\007\007\000\000\000\000\000\000\000\000\n\000\000\000\000\000\000\000\n\000\000\000\000\000\000\000\"\000\000\000\000\000\000\000" indirect_data: "<redacted>""cluster_id\n 9fb63a6e02234b7abb12f8e757bb6a25" } } } ops { id { term: 1 index: 3 } timestamp: 7316510920850120704 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\001\007\003\000\000\000\000\000\000\000\000\014\000\000\000\000\000\000\000\014\000\000\000\000\000\000\000d\003\000\000\000\000\000\000" indirect_data: "<redacted>""root-ca-info\n\320\0030\202\001\314\002\001\000\002a\000\343\327\233\341e\353\373\207\210t\035s5\200\316q\251\210\270FP\214\025Mq\177\016\203cD\247\265\'_\340\001\310d\001\312\031\326@\261\002\001\303\251\212.D\026\307Y\335c\035\233!.?\374\376\217\026\200a\220\025\251$\221Q\354\221\\\303Z\355\256\205\265M\363}\327\356J\027o3/\341\035\342\227\002\003\001\000\001\002a\000\305>\303\024>\310,\331\010\250}V_\333B\335\343\207\031\000\306z\177v\253\352\363\005\213\312\303\345pP\357\374\233\036\370\355\235\260~\020\277L\007\265\373\332\360\250\261\005\205,\255}\357\n\306\351,\rC\345\323\314\266+\271\324Y\360\371\310-\0034?es\3660\272\267$\372\010j\357\270\344_\343\001\0021\000\375\341\261\313\260\3164Ub\266\007x\030\307\242^6\375\341\337`\372\245}\331\331\330\230\272$|[:\256\022\345\325q\317(\222\027\242\236\267\326\224\243\0021\000\345\276J\363\000\367\264\364\252\254\362\371\352\236\255\206Gb\014\207\202\234\364\007\2574\220^65\334\0042e\032A\217\254K\253Y\333\334\020\220\261e}\0020}\257\t\221\311\t\214&\351#,7l-\010X\223\350\303\265\016\021\207s\3621Bd\371a\256\236\213B9\340\305\365\375\224\034\036\317F\273\303\342\371\0021\000\203\227e\222\236\301C2\310\367\333\014n\350\210w\3337;\371\372D0:\236\216;\n\333~*\021uT\027\237\351\315DbWK\351V\313\251\360y\0021\000\250\260\022t\014\214\014\201[\253\244\253\366(\032\340\205\021i$\352C\221?\357-\035\224ef\302n\336Y\214\314\321\316\215>\2271\204\324V\037\313\'\022\216\0030\202\001\2120\202\001\024\240\003\002\001\002\002\t\000\341\344\346t\033B\354y0\r\006\t*\206H\206\367\r\001\001\013\005\0000\0271\0250\023\006\003U\004\003\014\014kudu-ipki-ca0\036\027\r260809063909Z\027\r360806063909Z0\0271\0250\023\006\003U\004\003\014\014kudu-ipki-ca0|0\r\006\t*\206H\206\367\r\001\001\001\005\000\003k\0000h\002a\000\343\327\233\341e\353\373\207\210t\035s5\200\316q\251\210\270FP\214\025Mq\177\016\203cD\247\265\'_\340\001\310d\001\312\031\326@\261\002\001\303\251\212.D\026\307Y\335c\035\233!.?\374\376\217\026\200a\220\025\251$\221Q\354\221\\\303Z\355\256\205\265M\363}\327\356J\027o3/\341\035\342\227\002\003\001\000\001\243#0!0\016\006\003U\035\017\001\001\377\004\004\003\002\002\0040\017\006\003U\035\023\001\001\377\004\0050\003\001\001\3770\r\006\t*\206H\206\367\r\001\001\013\005\000\003a\000d\321\210\340\177\371\021\347\206m\3369\3116\306\242\314?5O\231\007\225\201\317E\357F|A\025\0307\345T3P\277E\323^[\206\366\262d\325\\14\003+r?\325F\306\3308\312\317CG\320\3667\257\226\320\361y\022UT=\n\213\276\000\233\206\373\215\312G\210\253z\271&\267\317~\213\257\005" } } } ops { id { term: 1 index: 4 } timestamp: 7316510920924729344 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\001\007\004\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000M\001\000\000\000\000\000\000" indirect_data: "<redacted>""\200\000\000\000\000\000\000\000\n\312\002\010\000\022\277\0020\202\001;\002\001\000\002A\000\321\360q\\%\014\267\271t\270 \321z\203A\024\323U\353x*\220\243`\250\336\301\272V(\\\326-\370\330\324\030\326,g\373\316\336Y\025a\213=\200\310\361\003\331\207h\1777\252Pm\0347\006\023\002\003\001\000\001\002@\027;\030\020.9^\0133\223<nq\264\034{\205w8{/\232\231\271\224v1\321\300\276\356\246\301U\206\373\016\223\316;\236\346A\262\021\204\351O\006\366\366\327S\tI\217\265\374`Yp\273?\241\002!\000\355;\230V\260\227B%\217\006\227\245\343\314\023\322\005\232\373\027=\301\333:\245s@\347\310;\006\275\002!\000\342\214\0324\021rH\276\200|\001\177\322\337\177xvN\3602\003\210\254\'\354L\225\010\277\326\265\017\002 U\217\331\370\342F\033\270Z&\201\211vh\203y\213F\003k\265}v\251gD\363R\345\227\314m\002!\000\205\007\350K3\003}\034\000\256\320\342\222\235\223h\325p=3\341G\\p\205\035\343V#\360\234\231\002!\000\304\364v6*\314\224<\203\"\rG\240s\014\244\275j-k\370QA\210\3416\320G_\210D\365\030\215\374\217\324\006" } } } ops { id { term: 1 index: 5 } timestamp: 7316510923066720256 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\001\007\001\000\000\000\000\000\000\000\000 \000\000\000\000\000\000\000 \000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\001\007\002\240\000\000\000\000\000\000\000 \000\000\000\000\000\000\000\300\000\000\000\000\000\000\000*\000\000\000\000\000\000\000" indirect_data: "<redacted>""2f0978615efc4192ab505b65958f78e1\n\rtest-workload\020\000\032P\n\025\010\n\022\003key\030\005 \001(\000@\000H\000P\000h\000\n\031\010\013\022\007int_val\030\005 \000(\000@\000H\000P\000h\000\n\034\010\014\022\nstring_val\030\010 \000(\001@\000H\000P\000h\000(\0010\002@\rJ\006\022\004\n\002\010\nP\215\301\340\323\006b\000j\005slave48ce55ef3b644f0bb384777b94214bdc \0002 2f0978615efc4192ab505b65958f78e1:\004\022\000\032\000" } } } ops { id { term: 1 index: 6 } timestamp: 7316510923097587712 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\002\007\002\000\000\000\000\000\000\000\000 \000\000\000\000\000\000\000 \000\000\000\000\000\000\000\232\000\000\000\000\000\000\000" indirect_data: "<redacted>""48ce55ef3b644f0bb384777b94214bdc\032J\010\000\032F\010\377\377\377\377\377\377\377\377\377\001\020\001\0327\n d496a732f4064e9a8d21acba3a5b3971\020\001\032\021\n\013127.6.111.1\020\341\320\002 \001*\"Sending initial creation of tablet2 2f0978615efc4192ab505b65958f78e1:\004\022\000\032\000" } } } ops { id { term: 1 index: 7 } timestamp: 7316510923324452864 op_type: WRITE_OP write_request { tablet_id: "00000000000000000000000000000000" schema { columns { name: "entry_type" type: INT8 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "entry_id" type: STRING is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "metadata" type: STRING is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } row_operations { rows: "<redacted>""\002\007\002\000\000\000\000\000\000\000\000 \000\000\000\000\000\000\000 \000\000\000\000\000\000\000\277\000\000\000\000\000\000\000" indirect_data: "<redacted>""48ce55ef3b644f0bb384777b94214bdc\032l\010\001\022 d496a732f4064e9a8d21acba3a5b3971\032F\010\377\377\377\377\377\377\377\377\377\001\020\001\0327\n d496a732f4064e9a8d21acba3a5b3971\020\001\032\021\n\013127.6.111.1\020\341\320\002 \002*%Tablet reported with an active leader2 2f0978615efc4192ab505b65958f78e1:\004\022\000\032\000" } } } ops { id { term: 1 index: 8 } timestamp: 7316510925730639872 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "00000000000000000000000000000000" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } } new_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } } } } dest_uuid: "0a127e3b50244511b0692b17845b3876" committed_index: 8 all_replicated_index: 0 safe_timestamp: 7316510925739094016 last_idx_appended_to_leader: 8
I20260809 06:39:12.329671 8205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:12.330099 7569 consensus_queue.cc:579] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: The tablet replica hosted on peer 0a127e3b50244511b0692b17845b3876 has failed
I20260809 06:39:12.330790 8030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:12.331053 8030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
W20260809 06:39:12.339075 8230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:12.339238 8230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:12.343427 8119 master.cc:537] Master@127.6.111.60:36913 shutting down...
I20260809 06:39:12.355013 8119 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 06:39:12.355505 8119 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 06:39:12.355690 8119 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: stopping tablet replica
I20260809 06:39:12.371136 8119 master.cc:552] Master@127.6.111.60:36913 shutdown complete.
Not implemented: Kudu only supports adding one master at a time; tentative Raft config doesn't match the UUIDs fetched from --master_addresses. Raft config + local UUID: [0a127e3b50244511b0692b17845b3876,1ace6e0f1df74902b713466fa639b837,3b5c031449f24ef3aee05c867d282717,e38e1cb910d54113b784b58b509ef675] vs fetched UUIDs: [1ace6e0f1df74902b713466fa639b837,3b5c031449f24ef3aee05c867d282717,e38e1cb910d54113b784b58b509ef675], diff: []
W20260809 06:39:12.810321 7569 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 -> Peer 0a127e3b50244511b0692b17845b3876 (127.6.111.60:36913): Couldn't send request to peer 0a127e3b50244511b0692b17845b3876. Error code: WRONG_SERVER_UUID (16). Status: Network error: Client connection negotiation failed: client connection to 127.6.111.60:36913: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260809 06:39:13.353958 8236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "0a127e3b50244511b0692b17845b3876" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:13.354302 8236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
W20260809 06:39:13.642304 8235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:13.642655 8235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:13.649775 8235 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20260809 06:39:13.690841 7589 catalog_manager.cc:6979] Initiating ChangeConfig request to remove master 127.6.111.60:36913: tablet_id: "00000000000000000000000000000000"
type: REMOVE_PEER
server {
permanent_uuid: "0a127e3b50244511b0692b17845b3876"
}
dest_uuid: "3b5c031449f24ef3aee05c867d282717"
cas_config_opid_index: 8
I20260809 06:39:13.691660 7589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } }
I20260809 06:39:13.693527 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 06:39:13.694069 8236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 06:39:13.696488 8030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 0a127e3b50244511b0692b17845b3876 (127.6.111.60) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:13.697139 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 0a127e3b50244511b0692b17845b3876 (127.6.111.60) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:13.699266 8030 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to remove master 127.6.111.60:36913
I20260809 06:39:13.699867 8236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:13.700090 8236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:13.700273 8235 tool_action_master.cc:566] Successfully removed master 127.6.111.60:36913 from the cluster. Please follow the next steps from the Kudu administration documentation on "Removing Kudu Masters from a Multi-Master Deployment" or "Recovering from a dead Kudu Master in a Multi-Master Deployment" as appropriate.
I20260809 06:39:13.700444 8237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:13.700762 8237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:13.703207 8268 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } }
I20260809 06:39:13.703505 8268 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:13.787863 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--webserver_interface=127.6.111.60
--webserver_port=42453
--builtin_ntp_servers=127.6.111.20:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--tablet_copy_fault_crash_during_download_wal=1
--tablet_copy_fault_crash_during_download_wal=0 with env {}
W20260809 06:39:14.122184 8271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:14.122452 8271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:14.122529 8271 flags.cc:425] Enabled unsafe flag: --tablet_copy_fault_crash_during_download_wal=0
W20260809 06:39:14.122577 8271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:14.129602 8271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:14.129695 8271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:14.129747 8271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:14.129789 8271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:14.138825 8271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:43033,127.6.111.61:33967,127.6.111.60:36913
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.111.60
--webserver_port=42453
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:14.140048 8271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:14.141886 8271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:14.154873 8278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:14.154911 8282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:14.154935 8279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:14.157462 8271 server_base.cc:1034] running on GCE node
I20260809 06:39:14.158215 8271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:14.159602 8271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:14.160831 8271 hybrid_clock.cc:648] HybridClock initialized: now 1786257554160782 us; error 41 us; skew 500 ppm
I20260809 06:39:14.161213 8271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:14.163569 8271 webserver.cc:460] Webserver started at http://127.6.111.60:42453/ using document root <none> and password file <none>
I20260809 06:39:14.164458 8271 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:14.164609 8271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:14.172526 8271 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:14.176082 8290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:14.177641 8271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:14.177806 8271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:14.178774 8271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:14.190652 8271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:14.191390 8271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:14.191689 8271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:14.210445 8271 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:36913
I20260809 06:39:14.210522 8356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:36913 every 8 connection(s)
I20260809 06:39:14.211922 8271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:14.218264 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8271
I20260809 06:39:14.218952 8357 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:14.224736 8357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap starting.
I20260809 06:39:14.233150 8357 log.cc:824] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:14.236037 8357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:14.236892 8357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap complete.
I20260809 06:39:14.245047 8357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:14.246013 8357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:14.246147 8357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ace6e0f1df74902b713466fa639b837, State: Initialized, Role: FOLLOWER
I20260809 06:39:14.246721 8357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } }
I20260809 06:39:14.248586 8365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } } }
I20260809 06:39:14.249137 8365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:14.249949 8357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:14.266023 8379 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:14.266255 8379 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:14.292925 8271 master_runner.cc:428] Detected that this master 1ace6e0f1df74902b713466fa639b837 is joining an existing cluster
I20260809 06:39:14.293109 8271 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.111.60:36913
I20260809 06:39:14.329274 7589 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.111.60:36913: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1ace6e0f1df74902b713466fa639b837"
member_type: NON_VOTER
last_known_addr {
host: "127.6.111.60"
port: 36913
}
attrs {
promote: true
}
}
dest_uuid: "3b5c031449f24ef3aee05c867d282717"
cas_config_opid_index: 9
I20260809 06:39:14.330163 7589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } }
I20260809 06:39:14.332500 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 06:39:14.332857 8325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:14.333407 8237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 06:39:14.335899 8237 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, NON_VOTER 1ace6e0f1df74902b713466fa639b837 (127.6.111.60) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.336740 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, NON_VOTER 1ace6e0f1df74902b713466fa639b837 (127.6.111.60) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.339918 8237 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.111.60:36913
I20260809 06:39:14.340284 8237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.340623 8237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:14.341245 8237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.341573 8237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:14.342044 8399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.342360 8399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:14.357818 8271 master.cc:537] Master@127.6.111.60:36913 shutting down...
I20260809 06:39:14.364537 8325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 06:39:14.366240 8237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:39:14.370723 7569 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 -> Peer 1ace6e0f1df74902b713466fa639b837 (127.6.111.60:36913): Couldn't send request to peer 1ace6e0f1df74902b713466fa639b837. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 06:39:14.373250 8271 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:39:14.373700 8271 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:39:14.373911 8271 tablet_replica.cc:330] stopping tablet replica
I20260809 06:39:14.390050 8271 master.cc:552] Master@127.6.111.60:36913 shutdown complete.
I20260809 06:39:14.390182 8271 master_runner.cc:305] Clearing existing system tablet
I20260809 06:39:14.390547 8271 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:14.390676 8271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:14.392910 8271 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 06:39:14.394618 8403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:14.394979 8271 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:14.395395 8271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 06:39:14.395558 8271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:14.399667 8271 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 06:39:14.402374 7589 master.cc:648] Network error: unable to get registration information for peer 1ace6e0f1df74902b713466fa639b837: Client connection negotiation failed: client connection to 127.6.111.60:36913: connect: Connection refused (error 111)
I20260809 06:39:14.408797 8271 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 06:39:14.408959 8271 log.cc:1196] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Deleting WAL directory at /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 06:39:14.409490 8271 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Deleting consensus metadata
I20260809 06:39:14.409755 8271 master_runner.cc:315] Copying system tablet from 127.6.111.61:33967
I20260809 06:39:14.412216 8271 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: tablet copy: Beginning tablet copy session from remote peer at address 127.6.111.61:33967
I20260809 06:39:14.419766 7611 tablet_copy_service.cc:140] P 3b5c031449f24ef3aee05c867d282717: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1ace6e0f1df74902b713466fa639b837 ({username='slave'} at 127.0.0.1:51948)
I20260809 06:39:14.419940 7611 tablet_copy_service.cc:161] P 3b5c031449f24ef3aee05c867d282717: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1ace6e0f1df74902b713466fa639b837 at {username='slave'} at 127.0.0.1:51948: session id = 1ace6e0f1df74902b713466fa639b837-00000000000000000000000000000000
I20260809 06:39:14.423592 7611 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 06:39:14.425742 8271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 06:39:14.432775 8271 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: tablet copy: Starting download of 0 data blocks...
I20260809 06:39:14.433094 8271 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: tablet copy: Starting download of 1 WAL segments...
I20260809 06:39:14.435726 8271 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 06:39:14.440001 7611 tablet_copy_service.cc:342] P 3b5c031449f24ef3aee05c867d282717: Request end of tablet copy session 1ace6e0f1df74902b713466fa639b837-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:51948
I20260809 06:39:14.440137 7611 tablet_copy_service.cc:433] P 3b5c031449f24ef3aee05c867d282717: ending tablet copy session 1ace6e0f1df74902b713466fa639b837-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1ace6e0f1df74902b713466fa639b837
I20260809 06:39:14.451680 8271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:14.454721 8414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:14.454826 8415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:14.455508 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:14.458725 8271 server_base.cc:1034] running on GCE node
I20260809 06:39:14.459045 8271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:14.459679 8271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:14.460915 8271 hybrid_clock.cc:648] HybridClock initialized: now 1786257554460869 us; error 42 us; skew 500 ppm
I20260809 06:39:14.461155 8271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:14.462833 8271 webserver.cc:460] Webserver started at http://127.6.111.60:42453/ using document root <none> and password file <none>
I20260809 06:39:14.463150 8271 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:14.463253 8271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:14.465711 8271 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 06:39:14.467515 8426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:14.467897 8271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 06:39:14.468043 8271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "1ace6e0f1df74902b713466fa639b837"
format_stamp: "Formatted at 2026-08-09 06:39:12 on dist-test-slave-bw88"
I20260809 06:39:14.468267 8271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:14.487393 8271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:14.488132 8271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:14.508118 8271 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:36913
I20260809 06:39:14.508143 8492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:36913 every 8 connection(s)
I20260809 06:39:14.509133 8271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.AutoAddMasterTest.TestFailTabletCopy.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:14.512768 8493 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:14.515126 8493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap starting.
I20260809 06:39:14.530573 8493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:14.531064 8493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: Bootstrap complete.
I20260809 06:39:14.531876 8493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } }
I20260809 06:39:14.532083 8493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1ace6e0f1df74902b713466fa639b837, State: Initialized, Role: LEARNER
I20260809 06:39:14.532383 8493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } }
I20260809 06:39:14.533262 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.533653 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:39:14.534452 8493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:14.557533 7589 master.cc:648] Service unavailable: unable to get registration information for peer 1ace6e0f1df74902b713466fa639b837: Catalog manager is not initialized. State: Starting
I20260809 06:39:14.562615 8511 catalog_manager.cc:1259] Loaded cluster ID: 9fb63a6e02234b7abb12f8e757bb6a25
I20260809 06:39:14.562737 8511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: loading cluster ID for follower catalog manager: success
I20260809 06:39:14.565397 8511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: acquiring CA information for follower catalog manager: success
I20260809 06:39:14.567423 8511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:39:14.980969 8461 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.10] Dedup: 1.10->[]
I20260809 06:39:14.981751 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5c031449f24ef3aee05c867d282717. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:14.982097 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:39:15.378832 8512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: true } } }
I20260809 06:39:15.379149 8512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:15.380203 8514 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717: attempting to promote NON_VOTER 1ace6e0f1df74902b713466fa639b837 to VOTER
I20260809 06:39:15.381003 8514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } }
I20260809 06:39:15.383124 8461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 LEARNER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 06:39:15.383193 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c031449f24ef3aee05c867d282717: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 06:39:15.383800 8516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 06:39:15.384209 8512 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 06:39:15.387063 8516 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 1ace6e0f1df74902b713466fa639b837 (127.6.111.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.387701 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 1ace6e0f1df74902b713466fa639b837 (127.6.111.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.389159 8461 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 1ace6e0f1df74902b713466fa639b837 (127.6.111.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.390344 8516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.390491 8512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.390647 8516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:15.390750 8512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:15.391029 8516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.391312 8516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5c031449f24ef3aee05c867d282717 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:15.393860 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.394170 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38e1cb910d54113b784b58b509ef675 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:15.394213 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b5c031449f24ef3aee05c867d282717" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e38e1cb910d54113b784b58b509ef675" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 43033 } } peers { permanent_uuid: "3b5c031449f24ef3aee05c867d282717" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 33967 } } peers { permanent_uuid: "1ace6e0f1df74902b713466fa639b837" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 36913 } attrs { promote: false } } }
I20260809 06:39:15.394482 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ace6e0f1df74902b713466fa639b837 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:19.362375 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7681
I20260809 06:39:19.375844 7832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:19.466499 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7681
I20260809 06:39:19.495643 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7449
I20260809 06:39:19.497004 7529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:19.609303 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7449
I20260809 06:39:19.639569 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 7538
I20260809 06:39:19.640969 7621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:19.751690 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 7538
I20260809 06:39:19.785656 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 8271
I20260809 06:39:19.786826 8486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:19.894260 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 8271
2026-08-09T06:39:19Z chronyd exiting
[ OK ] AutoAddMasterTest.TestFailTabletCopy (12278 ms)
[----------] 1 test from AutoAddMasterTest (12278 ms total)
[----------] 1 test from ParameterizedAddMasterTest
[ RUN ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/2
2026-08-09T06:39:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:19Z Disabled control of system clock
I20260809 06:39:19.975035 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:39435
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:39435,127.6.111.61:45875
--master_auto_join_cluster=false with env {}
W20260809 06:39:20.302042 8538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:20.302330 8538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:20.302407 8538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:20.309387 8538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:20.309494 8538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:20.309533 8538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:20.309595 8538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:20.318352 8538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:39435,127.6.111.61:45875
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:20.318684 8538 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:39435,127.6.111.61:45875'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:20.319705 8538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:20.321636 8538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:20.333518 8546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:20.333586 8547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:20.333557 8549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:20.334152 8538 server_base.cc:1034] running on GCE node
I20260809 06:39:20.334931 8538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:20.336308 8538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:20.337499 8538 hybrid_clock.cc:648] HybridClock initialized: now 1786257560337452 us; error 56 us; skew 500 ppm
I20260809 06:39:20.337859 8538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:20.340036 8538 webserver.cc:460] Webserver started at http://127.6.111.62:41445/ using document root <none> and password file <none>
I20260809 06:39:20.340704 8538 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:20.340847 8538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:20.341207 8538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:20.346275 8538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "01cf5ccc6e864fad86c2eda582b69553"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.347018 8538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "01cf5ccc6e864fad86c2eda582b69553"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.352212 8538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 06:39:20.355612 8557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:20.357002 8538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:20.357204 8538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "01cf5ccc6e864fad86c2eda582b69553"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.357455 8538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:20.368857 8538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:20.369742 8538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:20.370028 8538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:20.388782 8538 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:39435
I20260809 06:39:20.388815 8624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:39435 every 8 connection(s)
I20260809 06:39:20.390123 8538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:20.391994 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8538
I20260809 06:39:20.392205 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:20.393605 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:45875
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:39435,127.6.111.61:45875
--master_auto_join_cluster=false with env {}
I20260809 06:39:20.394089 8625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:20.400570 8625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:20.417816 8625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:39:20.419111 8564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
W20260809 06:39:20.420593 8625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:45875: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
I20260809 06:39:20.469936 8625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } attempt: 1
W20260809 06:39:20.471935 8625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:45875: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
I20260809 06:39:20.540282 8625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } attempt: 2
W20260809 06:39:20.542732 8625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:45875: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
I20260809 06:39:20.634073 8625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } attempt: 3
W20260809 06:39:20.636174 8625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:45875: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
W20260809 06:39:20.732987 8627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:20.733268 8627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:20.733359 8627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:20.740496 8627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:20.740598 8627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:20.740715 8627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:20.740821 8627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:20.750015 8627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:39435,127.6.111.61:45875
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:20.750398 8627 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:39435,127.6.111.61:45875'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:20.751519 8627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:20.753382 8627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:20.765527 8636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:20.765614 8642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:20.765636 8639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:20.766521 8627 server_base.cc:1034] running on GCE node
I20260809 06:39:20.767300 8627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:20.769318 8627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:20.770583 8627 hybrid_clock.cc:648] HybridClock initialized: now 1786257560770548 us; error 46 us; skew 500 ppm
I20260809 06:39:20.770959 8627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:20.773128 8627 webserver.cc:460] Webserver started at http://127.6.111.61:41489/ using document root <none> and password file <none>
I20260809 06:39:20.773908 8627 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:20.774021 8627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:20.774372 8627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:20.777032 8627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "13de037cec1e437089d863e57fc9b7e0"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.777901 8627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "13de037cec1e437089d863e57fc9b7e0"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.779475 8625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } attempt: 4
W20260809 06:39:20.781535 8625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:45875: Network error: Client connection negotiation failed: client connection to 127.6.111.61:45875: connect: Connection refused (error 111)
I20260809 06:39:20.783741 8627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:20.787127 8649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:20.788497 8627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:39:20.788689 8627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "13de037cec1e437089d863e57fc9b7e0"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:20.788899 8627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:20.799759 8627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:20.800611 8627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:20.800899 8627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:20.819918 8627 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:45875
I20260809 06:39:20.819943 8711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:45875 every 8 connection(s)
I20260809 06:39:20.821383 8627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:20.825295 8712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:20.826072 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8627
I20260809 06:39:20.826260 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:20.833443 8712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:20.845913 8712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:20.856444 8712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Bootstrap starting.
I20260809 06:39:20.860117 8712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:20.861303 8712 log.cc:824] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:20.863945 8712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: No bootstrap required, opened a new log
I20260809 06:39:20.872009 8712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:20.872342 8712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:20.872426 8712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13de037cec1e437089d863e57fc9b7e0, State: Initialized, Role: FOLLOWER
I20260809 06:39:20.873030 8712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:20.873987 8718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:20.874336 8718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:20.875136 8712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:20.884544 8737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:20.884677 8737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:20.986519 8718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:20.986871 8718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:20.988480 8718 leader_election.cc:288] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01cf5ccc6e864fad86c2eda582b69553 (127.6.111.62:39435)
W20260809 06:39:21.053202 8655 tablet.cc:2369] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:21.080737 8625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } attempt: 5
I20260809 06:39:21.094620 8625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Bootstrap starting.
I20260809 06:39:21.099022 8625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:21.100348 8625 log.cc:824] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:21.103269 8625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: No bootstrap required, opened a new log
I20260809 06:39:21.114424 8625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:21.114848 8625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:21.114948 8625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01cf5ccc6e864fad86c2eda582b69553, State: Initialized, Role: FOLLOWER
I20260809 06:39:21.115614 8625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:21.116907 8740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:21.117570 8740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:21.118219 8625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:21.118294 8591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13de037cec1e437089d863e57fc9b7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01cf5ccc6e864fad86c2eda582b69553" is_pre_election: true
I20260809 06:39:21.118923 8591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13de037cec1e437089d863e57fc9b7e0 in term 0.
I20260809 06:39:21.119810 8651 leader_election.cc:302] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0; no voters:
I20260809 06:39:21.120411 8718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:39:21.120645 8718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:21.120781 8718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 0 FOLLOWER]: Advancing to term 1
W20260809 06:39:21.122346 8566 tablet.cc:2369] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:21.124699 8718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:21.125330 8718 leader_election.cc:288] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 1 election: Requested vote from peers 01cf5ccc6e864fad86c2eda582b69553 (127.6.111.62:39435)
I20260809 06:39:21.127787 8591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13de037cec1e437089d863e57fc9b7e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01cf5ccc6e864fad86c2eda582b69553"
I20260809 06:39:21.128299 8591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 0 FOLLOWER]: Advancing to term 1
W20260809 06:39:21.131780 8752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:21.131915 8752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:21.131871 8591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13de037cec1e437089d863e57fc9b7e0 in term 1.
I20260809 06:39:21.132507 8651 leader_election.cc:302] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0; no voters:
I20260809 06:39:21.132861 8718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:21.137665 8718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 LEADER]: Becoming Leader. State: Replica: 13de037cec1e437089d863e57fc9b7e0, State: Running, Role: LEADER
I20260809 06:39:21.138253 8718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } }
I20260809 06:39:21.141314 8754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13de037cec1e437089d863e57fc9b7e0. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:21.141788 8754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:21.143918 8756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:21.146347 8756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:21.157104 8591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Refusing update from remote peer 13de037cec1e437089d863e57fc9b7e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:39:21.158272 8754 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:21.167706 8740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13de037cec1e437089d863e57fc9b7e0. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:21.167990 8740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:21.170149 8757 mvcc.cc:205] Tried to move back new op lower bound from 7316510970488238080 to 7316510970426601472. Current Snapshot: MvccSnapshot[applied={T|T < 7316510970488238080}]
I20260809 06:39:21.178464 8754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:21.178761 8754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:21.183735 8756 catalog_manager.cc:1347] Generated new cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:21.183868 8756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:21.187274 8740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } }
I20260809 06:39:21.187554 8740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:21.207201 8756 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:21.209127 8756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:21.221458 8756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Generated new TSK 0
I20260809 06:39:21.222632 8756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:21.249066 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:39435,127.6.111.61:45875
--builtin_ntp_servers=127.6.111.20:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:21.673091 8765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:21.673492 8765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:21.673666 8765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:21.681293 8765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:21.681531 8765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:21.690891 8765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:39435,127.6.111.61:45875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:21.692366 8765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:21.694419 8765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:21.708336 8770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:21.708297 8774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:21.708389 8772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:21.709539 8765 server_base.cc:1034] running on GCE node
I20260809 06:39:21.710225 8765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:21.711128 8765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:21.712381 8765 hybrid_clock.cc:648] HybridClock initialized: now 1786257561712323 us; error 35 us; skew 500 ppm
I20260809 06:39:21.712687 8765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:21.714931 8765 webserver.cc:460] Webserver started at http://127.6.111.1:43065/ using document root <none> and password file <none>
I20260809 06:39:21.715617 8765 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:21.715768 8765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:21.716089 8765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:21.718986 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "0e7fd5f5f0b64857b80983bfb25e219e"
format_stamp: "Formatted at 2026-08-09 06:39:21 on dist-test-slave-bw88"
I20260809 06:39:21.719821 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "0e7fd5f5f0b64857b80983bfb25e219e"
format_stamp: "Formatted at 2026-08-09 06:39:21 on dist-test-slave-bw88"
I20260809 06:39:21.725291 8765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 06:39:21.728696 8780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:21.730052 8765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:21.730238 8765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "0e7fd5f5f0b64857b80983bfb25e219e"
format_stamp: "Formatted at 2026-08-09 06:39:21 on dist-test-slave-bw88"
I20260809 06:39:21.730443 8765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:21.744669 8765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:21.745525 8765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:21.745811 8765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:21.746809 8765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:21.748221 8765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:21.748322 8765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:21.748467 8765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:21.748572 8765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:21.782945 8765 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:34777
I20260809 06:39:21.783004 8921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:34777 every 8 connection(s)
I20260809 06:39:21.784212 8765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:21.785557 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 8765
I20260809 06:39:21.785759 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:21.807382 8924 heartbeater.cc:347] Connected to a master server at 127.6.111.61:45875
I20260809 06:39:21.807410 8925 heartbeater.cc:347] Connected to a master server at 127.6.111.62:39435
I20260809 06:39:21.807693 8925 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:21.807696 8924 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:21.808532 8924 heartbeater.cc:511] Master 127.6.111.61:45875 requested a full tablet report, sending...
I20260809 06:39:21.808512 8925 heartbeater.cc:511] Master 127.6.111.62:39435 requested a full tablet report, sending...
I20260809 06:39:21.810527 8669 ts_manager.cc:194] Registered new tserver with Master: 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1:34777)
I20260809 06:39:21.810621 8579 ts_manager.cc:194] Registered new tserver with Master: 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1:34777)
I20260809 06:39:21.812163 8669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:59177
I20260809 06:39:21.820318 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:21.825714 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:21.847268 8669 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57200:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:21.879254 8843 tablet_service.cc:1467] Processing CreateTablet for tablet 97d13e403e7545e19713960f526b7872 (DEFAULT_TABLE table=first_table [id=46993ea521294f0294cca2c23e4c27b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:21.880597 8843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d13e403e7545e19713960f526b7872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:21.891912 8949 tablet_bootstrap.cc:492] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:21.895069 8949 tablet_bootstrap.cc:654] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:21.896248 8949 log.cc:824] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:21.898787 8949 tablet_bootstrap.cc:492] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: No bootstrap required, opened a new log
I20260809 06:39:21.899065 8949 ts_tablet_manager.cc:1397] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 06:39:21.906525 8949 raft_consensus.cc:348] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:21.906873 8949 raft_consensus.cc:374] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:21.906981 8949 raft_consensus.cc:729] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:21.907526 8949 consensus_queue.cc:260] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:21.907768 8949 raft_consensus.cc:388] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:21.907912 8949 raft_consensus.cc:482] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:21.908092 8949 raft_consensus.cc:3037] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:21.910087 8949 raft_consensus.cc:504] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:21.910565 8949 leader_election.cc:302] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:21.911144 8949 leader_election.cc:288] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:21.911252 8951 raft_consensus.cc:2781] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:21.912027 8951 raft_consensus.cc:686] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:21.912317 8924 heartbeater.cc:503] Master 127.6.111.61:45875 was elected leader, sending a full tablet report...
I20260809 06:39:21.912633 8951 consensus_queue.cc:237] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:21.913069 8949 ts_tablet_manager.cc:1428] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 06:39:21.918738 8669 catalog_manager.cc:5495] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1). New cstate: current_term: 1 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:22.134709 8752 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:22.134862 8752 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: loading cluster ID for follower catalog manager: success
I20260809 06:39:22.137681 8752 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: acquiring CA information for follower catalog manager: success
I20260809 06:39:22.139569 8752 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:39:22.873701 8669 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.111.60:40693: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
member_type: NON_VOTER
last_known_addr {
host: "127.6.111.60"
port: 40693
}
attrs {
promote: true
}
}
dest_uuid: "13de037cec1e437089d863e57fc9b7e0"
cas_config_opid_index: -1
I20260809 06:39:22.874794 8669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } }
I20260809 06:39:22.877278 8591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Refusing update from remote peer 13de037cec1e437089d863e57fc9b7e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 06:39:22.878033 8944 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 06:39:22.881312 9084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:22.881851 8944 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:22.881942 8591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index -1 to 8, NON_VOTER 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:22.887517 9084 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.111.60:40693
I20260809 06:39:22.890013 8944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:22.890251 8944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:22.897841 9085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:22.898355 9085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:22.903083 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:22.903509 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:23.422006 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: NON_VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: true } } }
I20260809 06:39:23.422320 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:23.423512 9107 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: attempting to promote NON_VOTER 4c9d14f63b9445498f59d8cfbe5158ea to VOTER
I20260809 06:39:23.424258 9107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:23.426357 8591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Refusing update from remote peer 13de037cec1e437089d863e57fc9b7e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 06:39:23.426949 9088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 06:39:23.427345 9106 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 06:39:23.430037 9107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.430645 8591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.432850 9107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.433008 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.433409 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:23.433617 9107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:23.433907 9107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.434235 9107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:23.435252 9112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:23.435606 9112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 06:39:23.992930 8653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.60:40693: connect: Connection refused (error 111)
W20260809 06:39:23.994510 8653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 -> Peer 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693): Couldn't send request to peer 4c9d14f63b9445498f59d8cfbe5158ea. Status: Network error: Client connection negotiation failed: client connection to 127.6.111.60:40693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 06:39:24.408712 9136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:24.409075 9136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
W20260809 06:39:24.654165 8669 master_service.cc:291] Master 127.6.111.60:40693 already present
I20260809 06:39:25.238336 9136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:25.238792 9136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
W20260809 06:39:25.251950 8653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 -> Peer 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693): Couldn't send request to peer 4c9d14f63b9445498f59d8cfbe5158ea. Status: Network error: Client connection negotiation failed: client connection to 127.6.111.60:40693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 06:39:25.623163 6588 dynamic_multi_master-test.cc:554] Add master stderr: W20260809 06:39:25.495865 9299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:25.496201 9299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:25.496412 9299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:25.503314 9299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:25.503449 9299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:25.503563 9299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:25.503674 9299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:39:25.503803 9299 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Illegal state: Failed bringing up new master. See Kudu Master log for details.: Master 127.6.111.62:39435 already running
I20260809 06:39:25.626490 6588 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:25.626618 6588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:25.628918 6588 dynamic_multi_master-test.cc:645] Shutting down the old cluster
I20260809 06:39:25.629034 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 8765
I20260809 06:39:25.632344 9300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13de037cec1e437089d863e57fc9b7e0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:25.632712 9300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:25.637501 8915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:25.725310 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 8765
I20260809 06:39:25.752650 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 8538
I20260809 06:39:25.753870 8617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:25.863282 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 8538
I20260809 06:39:25.891950 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 8627
I20260809 06:39:25.893000 8704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:26.005664 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 8627
2026-08-09T06:39:26Z chronyd exiting
I20260809 06:39:26.048517 6588 dynamic_multi_master-test.cc:648] Bringing up the migrated cluster
2026-08-09T06:39:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:26Z Disabled control of system clock
I20260809 06:39:26.078063 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:39435
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:36491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false with env {}
W20260809 06:39:26.414115 9333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:26.414427 9333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:26.414506 9333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:26.421602 9333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:26.421708 9333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:26.421751 9333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:26.421808 9333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:26.431263 9333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:26.432725 9333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:26.434587 9333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:26.447052 9342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:26.447121 9344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:26.447081 9341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:26.448035 9333 server_base.cc:1034] running on GCE node
I20260809 06:39:26.448896 9333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:26.450453 9333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:26.451679 9333 hybrid_clock.cc:648] HybridClock initialized: now 1786257566451619 us; error 31 us; skew 500 ppm
I20260809 06:39:26.452026 9333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:26.454413 9333 webserver.cc:460] Webserver started at http://127.6.111.62:38049/ using document root <none> and password file <none>
I20260809 06:39:26.455191 9333 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:26.455358 9333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:26.460795 9333 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:26.463753 9351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:26.465025 9333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:26.465224 9333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "01cf5ccc6e864fad86c2eda582b69553"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:26.466262 9333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:26.511227 9333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:26.512172 9333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:26.512491 9333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:26.531672 9333 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:39435
I20260809 06:39:26.531723 9406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:39435 every 8 connection(s)
I20260809 06:39:26.533152 9333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:26.536525 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 9333
I20260809 06:39:26.536731 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:26.537889 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:45875
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:36491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false with env {}
I20260809 06:39:26.540148 9407 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:26.543536 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Bootstrap starting.
I20260809 06:39:26.567520 9407 log.cc:824] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:26.582597 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:26.583298 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Bootstrap complete.
I20260809 06:39:26.596961 9407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:26.598383 9407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01cf5ccc6e864fad86c2eda582b69553, State: Initialized, Role: FOLLOWER
I20260809 06:39:26.599129 9407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:26.600121 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:26.600564 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:26.601435 9407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:26.611275 9425 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:26.611397 9425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: loading cluster ID for follower catalog manager: success
I20260809 06:39:26.614270 9425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: acquiring CA information for follower catalog manager: success
I20260809 06:39:26.616170 9425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 06:39:26.914736 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:26.915023 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:26.915153 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:26.922477 9409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:26.922574 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:26.922652 9409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:26.922760 9409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:26.931831 9409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:26.933164 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:26.934986 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:26.947000 9433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:26.946879 9434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:26.947541 9436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:26.948063 9409 server_base.cc:1034] running on GCE node
I20260809 06:39:26.948827 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:26.950270 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:26.951519 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786257566951435 us; error 57 us; skew 500 ppm
I20260809 06:39:26.951869 9409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:26.954278 9409 webserver.cc:460] Webserver started at http://127.6.111.61:33993/ using document root <none> and password file <none>
I20260809 06:39:26.954967 9409 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:26.955116 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:26.960469 9409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:26.963711 9443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:26.965167 9409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:26.965335 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "13de037cec1e437089d863e57fc9b7e0"
format_stamp: "Formatted at 2026-08-09 06:39:20 on dist-test-slave-bw88"
I20260809 06:39:26.966337 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:26.984055 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:26.984841 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:26.985095 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:27.003074 9409 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:45875
I20260809 06:39:27.003088 9504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:45875 every 8 connection(s)
I20260809 06:39:27.004700 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:27.011485 9505 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:27.012900 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 9409
I20260809 06:39:27.013099 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:27.014119 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:40693
--webserver_interface=127.6.111.60
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:36491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false with env {}
I20260809 06:39:27.015200 9505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Bootstrap starting.
I20260809 06:39:27.032832 9505 log.cc:824] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:27.044366 9505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:27.044893 9505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: Bootstrap complete.
I20260809 06:39:27.054124 9505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:27.054961 9505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13de037cec1e437089d863e57fc9b7e0, State: Initialized, Role: FOLLOWER
I20260809 06:39:27.055565 9505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:27.056408 9512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:27.056871 9512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:27.057610 9505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:27.067087 9524 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:27.067202 9524 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: loading cluster ID for follower catalog manager: success
I20260809 06:39:27.070468 9524 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: acquiring CA information for follower catalog manager: success
I20260809 06:39:27.072453 9524 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 06:39:27.386420 9508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:27.386713 9508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:27.386843 9508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:27.393874 9508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:27.393977 9508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:27.394073 9508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:27.394191 9508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:27.403281 9508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.60:40693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.111.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:27.404632 9508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:27.406576 9508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:27.419616 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:27.419737 9531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:27.420239 9535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:27.420603 9508 server_base.cc:1034] running on GCE node
I20260809 06:39:27.421494 9508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:27.422964 9508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:27.424214 9508 hybrid_clock.cc:648] HybridClock initialized: now 1786257567424141 us; error 52 us; skew 500 ppm
I20260809 06:39:27.424573 9508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:27.427017 9508 webserver.cc:460] Webserver started at http://127.6.111.60:38059/ using document root <none> and password file <none>
I20260809 06:39:27.427727 9508 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:27.427878 9508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:27.433506 9508 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:27.436563 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:27.438081 9508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:27.438280 9508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal
uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
format_stamp: "Formatted at 2026-08-09 06:39:22 on dist-test-slave-bw88"
I20260809 06:39:27.439260 9508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:27.454046 9508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:27.454843 9508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:27.455135 9508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:27.474112 9508 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.60:40693
I20260809 06:39:27.474144 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.60:40693 every 8 connection(s)
I20260809 06:39:27.475562 9508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb
I20260809 06:39:27.476691 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 9508
I20260809 06:39:27.476882 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/wal/instance
I20260809 06:39:27.484642 9606 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:27.488832 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: Bootstrap starting.
I20260809 06:39:27.510237 9606 log.cc:824] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:27.522205 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:27.522677 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: Bootstrap complete.
I20260809 06:39:27.530968 9606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:27.532224 9606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9d14f63b9445498f59d8cfbe5158ea, State: Initialized, Role: FOLLOWER
I20260809 06:39:27.532881 9606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:27.533947 9613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:27.534344 9613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:27.535117 9606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:27.546084 9626 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:27.546183 9626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: loading cluster ID for follower catalog manager: success
I20260809 06:39:27.549927 9626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: acquiring CA information for follower catalog manager: success
I20260809 06:39:27.552543 9626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:39:28.300837 9631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:28.301208 9631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:28.303483 9631 leader_election.cc:288] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:28.311975 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "13de037cec1e437089d863e57fc9b7e0" is_pre_election: true
I20260809 06:39:28.312175 9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" is_pre_election: true
I20260809 06:39:28.312628 9475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 1.
I20260809 06:39:28.312667 9575 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 1.
I20260809 06:39:28.313417 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0; no voters:
I20260809 06:39:28.313747 9631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 06:39:28.313905 9631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:28.314033 9631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:28.317934 9631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:28.318459 9631 leader_election.cc:288] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 2 election: Requested vote from peers 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:28.318976 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "13de037cec1e437089d863e57fc9b7e0"
I20260809 06:39:28.319043 9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
I20260809 06:39:28.319186 9475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:28.319252 9575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:28.322927 9475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 2.
I20260809 06:39:28.322926 9575 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 2.
I20260809 06:39:28.323338 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 4c9d14f63b9445498f59d8cfbe5158ea; no voters:
I20260809 06:39:28.323640 9631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:39:28.324035 9631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 LEADER]: Becoming Leader. State: Replica: 01cf5ccc6e864fad86c2eda582b69553, State: Running, Role: LEADER
I20260809 06:39:28.324498 9631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:28.327133 9638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.327423 9638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:28.328130 9642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:28.330566 9642 catalog_manager.cc:669] Loaded metadata for table first_table [id=46993ea521294f0294cca2c23e4c27b8]
I20260809 06:39:28.332360 9642 tablet_loader.cc:96] loaded metadata for tablet 97d13e403e7545e19713960f526b7872 (table first_table [id=46993ea521294f0294cca2c23e4c27b8])
I20260809 06:39:28.332674 9642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:28.333534 9642 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:28.333630 9642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:28.335063 9642 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:28.335984 9642 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Loaded TSK: 0
I20260809 06:39:28.336840 9642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:28.378058 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--builtin_ntp_servers=127.6.111.20:36491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:28.720010 9643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:28.720261 9643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:28.720338 9643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:28.727205 9643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:28.727345 9643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:28.736148 9643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:39435,127.6.111.61:45875,127.6.111.60:40693
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:28.737454 9643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:28.739342 9643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:28.752674 9651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:28.752758 9648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:28.752741 9649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:28.757815 9643 server_base.cc:1034] running on GCE node
I20260809 06:39:28.758342 9643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:28.759356 9643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:28.760620 9643 hybrid_clock.cc:648] HybridClock initialized: now 1786257568760504 us; error 77 us; skew 500 ppm
I20260809 06:39:28.760915 9643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:28.763142 9643 webserver.cc:460] Webserver started at http://127.6.111.1:33221/ using document root <none> and password file <none>
I20260809 06:39:28.763809 9643 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:28.763979 9643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:28.769379 9643 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:28.769529 9575 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 2 FOLLOWER]: Refusing update from remote peer 01cf5ccc6e864fad86c2eda582b69553: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260809 06:39:28.770592 9638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 06:39:28.772763 9658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:28.774192 9643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:28.774469 9657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.774778 9657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:28.774361 9643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "0e7fd5f5f0b64857b80983bfb25e219e"
format_stamp: "Formatted at 2026-08-09 06:39:21 on dist-test-slave-bw88"
I20260809 06:39:28.775509 9643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:28.776746 9638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.777093 9638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:28.778576 9475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 2 FOLLOWER]: Refusing update from remote peer 01cf5ccc6e864fad86c2eda582b69553: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20260809 06:39:28.779564 9638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 06:39:28.785251 9657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.785600 9657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:28.788610 9668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.789273 9668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:28.789280 9664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.789794 9664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:28.799841 9643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:28.800631 9643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:28.800911 9643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:28.801980 9643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:28.803942 9676 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 06:39:28.809237 9643 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 06:39:28.809378 9643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 06:39:28.809535 9643 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 06:39:28.812707 9643 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 06:39:28.812777 9643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 06:39:28.813155 9676 tablet_bootstrap.cc:492] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:28.835181 9676 log.cc:824] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:28.840263 9676 tablet_bootstrap.cc:492] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:28.841279 9676 tablet_bootstrap.cc:492] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap complete.
I20260809 06:39:28.843394 9676 ts_tablet_manager.cc:1397] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.030s user 0.020s sys 0.008s
I20260809 06:39:28.853540 9643 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:42845
I20260809 06:39:28.853596 9807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:42845 every 8 connection(s)
I20260809 06:39:28.855278 9643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:28.860657 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 9643
I20260809 06:39:28.860920 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:28.874370 9676 raft_consensus.cc:348] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:28.876602 9676 raft_consensus.cc:729] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:28.877528 9676 consensus_queue.cc:260] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:28.877905 9676 raft_consensus.cc:388] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:28.878089 9676 raft_consensus.cc:482] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:28.878306 9676 raft_consensus.cc:3037] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:28.892156 9676 raft_consensus.cc:504] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:28.893461 9676 leader_election.cc:302] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:28.898524 9810 heartbeater.cc:347] Connected to a master server at 127.6.111.62:39435
I20260809 06:39:28.898845 9809 heartbeater.cc:347] Connected to a master server at 127.6.111.60:40693
I20260809 06:39:28.898869 9810 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:28.899001 9809 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:28.900125 9810 heartbeater.cc:511] Master 127.6.111.62:39435 requested a full tablet report, sending...
I20260809 06:39:28.900174 9809 heartbeater.cc:511] Master 127.6.111.60:40693 requested a full tablet report, sending...
I20260809 06:39:28.903717 9369 ts_manager.cc:194] Registered new tserver with Master: 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1:42845)
I20260809 06:39:28.904340 9676 leader_election.cc:288] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:39:28.904803 9820 raft_consensus.cc:2781] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:39:28.906208 9820 raft_consensus.cc:686] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [term 2 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:28.906977 9369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:49233
I20260809 06:39:28.906980 9820 consensus_queue.cc:237] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } }
I20260809 06:39:28.910064 9564 ts_manager.cc:194] Registered new tserver with Master: 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1:42845)
I20260809 06:39:28.911085 9810 heartbeater.cc:503] Master 127.6.111.62:39435 was elected leader, sending a full tablet report...
I20260809 06:39:28.911586 9811 heartbeater.cc:347] Connected to a master server at 127.6.111.61:45875
I20260809 06:39:28.911706 9811 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:28.912097 9811 heartbeater.cc:511] Master 127.6.111.61:45875 requested a full tablet report, sending...
I20260809 06:39:28.912796 9676 ts_tablet_manager.cc:1428] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.069s user 0.032s sys 0.016s
I20260809 06:39:28.916118 9369 catalog_manager.cc:5495] T 97d13e403e7545e19713960f526b7872 P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 34777 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:28.917073 9462 ts_manager.cc:194] Registered new tserver with Master: 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1:42845)
I20260809 06:39:28.924712 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:28.926939 9662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:28.927245 9662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:28.955144 6588 dynamic_multi_master-test.cc:680] Transferring leadership to master: 4c9d14f63b9445498f59d8cfbe5158ea
I20260809 06:39:28.958551 9380 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "01cf5ccc6e864fad86c2eda582b69553"
mode: GRACEFUL
new_leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
from {username='slave'} at 127.0.0.1:49798
I20260809 06:39:28.958900 9380 raft_consensus.cc:595] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 LEADER]: Received request to transfer leadership to TS 4c9d14f63b9445498f59d8cfbe5158ea
I20260809 06:39:29.418335 9631 raft_consensus.cc:979] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: : Instructing follower 4c9d14f63b9445498f59d8cfbe5158ea to start an election
I20260809 06:39:29.418541 9638 raft_consensus.cc:1067] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 LEADER]: Signalling peer 4c9d14f63b9445498f59d8cfbe5158ea to start an election
I20260809 06:39:29.419334 9575 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
from {username='slave'} at 127.0.0.1:54458
I20260809 06:39:29.419575 9575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 06:39:29.419688 9575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:39:29.421614 9575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:29.423607 9575 leader_election.cc:288] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [CANDIDATE]: Term 3 election: Requested vote from peers 01cf5ccc6e864fad86c2eda582b69553 (127.6.111.62:39435), 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875)
I20260809 06:39:29.431305 9380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "01cf5ccc6e864fad86c2eda582b69553"
I20260809 06:39:29.431655 9380 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 06:39:29.431766 9380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01cf5ccc6e864fad86c2eda582b69553, State: Running, Role: LEADER
I20260809 06:39:29.432111 9380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:29.432322 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "13de037cec1e437089d863e57fc9b7e0"
I20260809 06:39:29.432534 9475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:39:29.432843 9380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:39:29.435448 9475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9d14f63b9445498f59d8cfbe5158ea in term 3.
I20260809 06:39:29.435626 9380 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9d14f63b9445498f59d8cfbe5158ea in term 3.
I20260809 06:39:29.436273 9548 leader_election.cc:302] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0, 4c9d14f63b9445498f59d8cfbe5158ea; no voters:
I20260809 06:39:29.436929 9838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 FOLLOWER]: Leader election won for term 3
I20260809 06:39:29.437304 9838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 LEADER]: Becoming Leader. State: Replica: 4c9d14f63b9445498f59d8cfbe5158ea, State: Running, Role: LEADER
I20260809 06:39:29.437891 9838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:29.439288 9839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c9d14f63b9445498f59d8cfbe5158ea. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.439683 9839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:29.440299 9840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:29.442763 9840 catalog_manager.cc:669] Loaded metadata for table first_table [id=46993ea521294f0294cca2c23e4c27b8]
I20260809 06:39:29.444334 9840 tablet_loader.cc:96] loaded metadata for tablet 97d13e403e7545e19713960f526b7872 (table first_table [id=46993ea521294f0294cca2c23e4c27b8])
I20260809 06:39:29.444621 9840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:29.445521 9840 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:29.445612 9840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:29.446871 9840 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:29.447731 9840 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea: Loaded TSK: 0
I20260809 06:39:29.448576 9840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:29.644448 9723 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
01cf5ccc6e864fad86c2eda582b69553 | 127.6.111.62:39435 | HEALTHY
13de037cec1e437089d863e57fc9b7e0 | 127.6.111.61:45875 | HEALTHY
4c9d14f63b9445498f59d8cfbe5158ea | 127.6.111.60:40693 | HEALTHY
All reported replicas are:
A = 01cf5ccc6e864fad86c2eda582b69553
B = 13de037cec1e437089d863e57fc9b7e0
C = 4c9d14f63b9445498f59d8cfbe5158ea
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
A | A B C | 3 | 9 | Yes
B | A B C | 3 | 9 | Yes
C | A B C* | 3 | 9 | Yes
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb | hidden | 127.6.111.62:39435
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb | hidden | 127.6.111.61:45875
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb | hidden | 127.6.111.60:40693
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36491 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
0e7fd5f5f0b64857b80983bfb25e219e | 127.6.111.1:42845 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.6.111.1 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36491 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: master consensus error: there are master consensus conflicts
FAILED
I20260809 06:39:29.676265 6588 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260809 06:39:29.819413 9380 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Refusing update from remote peer 4c9d14f63b9445498f59d8cfbe5158ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260809 06:39:29.820277 9839 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260809 06:39:29.821991 9638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c9d14f63b9445498f59d8cfbe5158ea. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.822293 9638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:29.823797 9839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.824080 9839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:29.824421 9475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Refusing update from remote peer 4c9d14f63b9445498f59d8cfbe5158ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20260809 06:39:29.824776 9638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.825037 9638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:29.825516 9838 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260809 06:39:29.827821 9889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.828305 9889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:29.828141 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c9d14f63b9445498f59d8cfbe5158ea. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:29.828516 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:29.928263 9723 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
01cf5ccc6e864fad86c2eda582b69553 | 127.6.111.62:39435 | HEALTHY
13de037cec1e437089d863e57fc9b7e0 | 127.6.111.61:45875 | HEALTHY
4c9d14f63b9445498f59d8cfbe5158ea | 127.6.111.60:40693 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 3 server(s) checked
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_reuseport | true | experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb | hidden | 127.6.111.62:39435
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb | hidden | 127.6.111.61:45875
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/master-2/data/info.pb | hidden | 127.6.111.60:40693
tsk_num_rsa_bits | 512 | experimental | all 3 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36491 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
0e7fd5f5f0b64857b80983bfb25e219e | 127.6.111.1:42845 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.6.111.1 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL_2.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36491 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 3
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260809 06:39:29.959184 6588 log_verifier.cc:126] Checking tablet 97d13e403e7545e19713960f526b7872
I20260809 06:39:29.970633 6588 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 97d13e403e7545e19713960f526b7872
I20260809 06:39:29.970881 6588 dynamic_multi_master-test.cc:688] Verifying the first table
I20260809 06:39:30.005744 6588 dynamic_multi_master-test.cc:691] Creating and verifying the second table
I20260809 06:39:30.044293 9563 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54538:
name: "second_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:30.050411 9885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:30.050695 9885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:30.076598 9723 tablet_service.cc:1467] Processing CreateTablet for tablet 63886e36eac54288a8088e5b07b6de8d (DEFAULT_TABLE table=second_table [id=dde222fb736a4beea0bf48fd5b9363f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:30.077517 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63886e36eac54288a8088e5b07b6de8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:30.084249 9961 tablet_bootstrap.cc:492] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:30.086347 9961 tablet_bootstrap.cc:654] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:30.088419 9961 tablet_bootstrap.cc:492] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e: No bootstrap required, opened a new log
I20260809 06:39:30.088579 9961 ts_tablet_manager.cc:1397] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:30.089522 9961 raft_consensus.cc:348] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.089713 9961 raft_consensus.cc:374] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:30.089840 9961 raft_consensus.cc:729] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:30.090109 9961 consensus_queue.cc:260] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.090319 9961 raft_consensus.cc:388] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:30.090412 9961 raft_consensus.cc:482] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:30.090583 9961 raft_consensus.cc:3037] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:30.092470 9961 raft_consensus.cc:504] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.092728 9961 leader_election.cc:302] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:30.097612 9961 leader_election.cc:288] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:30.097775 9963 raft_consensus.cc:2781] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:30.098505 9809 heartbeater.cc:503] Master 127.6.111.60:40693 was elected leader, sending a full tablet report...
I20260809 06:39:30.098557 9961 ts_tablet_manager.cc:1428] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 06:39:30.099267 9963 raft_consensus.cc:686] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:30.099613 9963 consensus_queue.cc:237] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.103471 9563 catalog_manager.cc:5495] T 63886e36eac54288a8088e5b07b6de8d P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1). New cstate: current_term: 1 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:30.148814 6588 dynamic_multi_master-test.cc:700] Pausing and resuming individual masters
I20260809 06:39:30.148934 6588 dynamic_multi_master-test.cc:704] Pausing master 01cf5ccc6e864fad86c2eda582b69553, 127.6.111.62:39435
I20260809 06:39:30.207170 9563 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54550:
name: "table-0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:30.218437 9723 tablet_service.cc:1467] Processing CreateTablet for tablet db9487eaa2c44c5b8683ad4f4e0924eb (DEFAULT_TABLE table=table-0 [id=91e6e5c7333e4d229498d98a023a507c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:30.219050 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9487eaa2c44c5b8683ad4f4e0924eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:30.224038 9961 tablet_bootstrap.cc:492] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:30.225817 9961 tablet_bootstrap.cc:654] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:30.227885 9961 tablet_bootstrap.cc:492] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e: No bootstrap required, opened a new log
I20260809 06:39:30.228089 9961 ts_tablet_manager.cc:1397] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:30.228933 9961 raft_consensus.cc:348] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.229154 9961 raft_consensus.cc:374] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:30.229250 9961 raft_consensus.cc:729] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:30.229524 9961 consensus_queue.cc:260] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.229712 9961 raft_consensus.cc:388] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:30.229839 9961 raft_consensus.cc:482] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:30.229954 9961 raft_consensus.cc:3037] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:30.231832 9961 raft_consensus.cc:504] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.232095 9961 leader_election.cc:302] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:30.232359 9963 raft_consensus.cc:2781] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:30.232672 9963 raft_consensus.cc:686] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:30.232928 9963 consensus_queue.cc:237] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.233857 9961 leader_election.cc:288] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:30.236270 9961 ts_tablet_manager.cc:1428] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 06:39:30.236559 9563 catalog_manager.cc:5495] T db9487eaa2c44c5b8683ad4f4e0924eb P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1). New cstate: current_term: 1 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:30.252605 6588 dynamic_multi_master-test.cc:704] Pausing master 13de037cec1e437089d863e57fc9b7e0, 127.6.111.61:45875
I20260809 06:39:30.321645 9563 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54568:
name: "table-1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:30.333753 9723 tablet_service.cc:1467] Processing CreateTablet for tablet d2eb6bd665ec4648809382e9bb656422 (DEFAULT_TABLE table=table-1 [id=f83623fa273f450fbd3336ba464fd98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:30.334211 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2eb6bd665ec4648809382e9bb656422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:30.340034 9961 tablet_bootstrap.cc:492] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:30.341650 9961 tablet_bootstrap.cc:654] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:30.343961 9961 tablet_bootstrap.cc:492] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e: No bootstrap required, opened a new log
I20260809 06:39:30.344079 9961 ts_tablet_manager.cc:1397] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:30.344694 9961 raft_consensus.cc:348] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.344864 9961 raft_consensus.cc:374] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:30.344995 9961 raft_consensus.cc:729] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:30.345244 9961 consensus_queue.cc:260] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.345484 9961 raft_consensus.cc:388] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:30.345602 9961 raft_consensus.cc:482] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:30.345734 9961 raft_consensus.cc:3037] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:30.348508 9961 raft_consensus.cc:504] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.348805 9961 leader_election.cc:302] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:30.348996 9961 leader_election.cc:288] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:30.349112 9963 raft_consensus.cc:2781] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:30.349263 9963 raft_consensus.cc:686] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:30.349578 9963 consensus_queue.cc:237] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:30.349606 9961 ts_tablet_manager.cc:1428] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 06:39:30.352309 9563 catalog_manager.cc:5495] T d2eb6bd665ec4648809382e9bb656422 P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1). New cstate: current_term: 1 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:30.363909 6588 dynamic_multi_master-test.cc:704] Pausing master 4c9d14f63b9445498f59d8cfbe5158ea, 127.6.111.60:40693
I20260809 06:39:31.858347 10039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4c9d14f63b9445498f59d8cfbe5158ea)
I20260809 06:39:31.858589 10039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:31.859270 10039 leader_election.cc:288] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:31.859967 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "13de037cec1e437089d863e57fc9b7e0" is_pre_election: true
I20260809 06:39:31.866317 10040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4c9d14f63b9445498f59d8cfbe5158ea)
I20260809 06:39:31.866499 10040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:31.868614 10040 leader_election.cc:288] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01cf5ccc6e864fad86c2eda582b69553 (127.6.111.62:39435), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:31.875905 9380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13de037cec1e437089d863e57fc9b7e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: false dest_uuid: "01cf5ccc6e864fad86c2eda582b69553" is_pre_election: true
I20260809 06:39:31.876269 9380 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13de037cec1e437089d863e57fc9b7e0 for term 4 because replica has last-logged OpId of term: 3 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
W20260809 06:39:33.415040 10036 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:30.385370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.111.60:40693 (local address 127.0.0.1:54570)
0809 06:39:30.391765 (+ 6395us) negotiation.cc:107] Waiting for socket to connect
0809 06:39:30.391780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:39:30.391859 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:39:33.413771 (+3021912us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.6.111.60:40693: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":6265,"thread_start_us":6560,"threads_started":1}
W20260809 06:39:34.003330 9355 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.111.60:40693 timed out after 2.143s (SENT)
W20260809 06:39:34.007201 9447 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.6.111.60:40693 for RPC RequestConsensusVote timed out after 2.137s (ON_OUTBOUND_QUEUE)
W20260809 06:39:34.950613 10046 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:31.869212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.111.60:40693 (local address 127.0.0.1:54576)
0809 06:39:31.870079 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 06:39:31.870128 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 06:39:31.870308 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:39:34.949677 (+3079369us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.6.111.60:40693: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":25,"thread_start_us":607,"threads_started":1}
W20260809 06:39:34.950968 9447 leader_election.cc:334] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693): Timed out: Client connection negotiation failed: client connection to 127.6.111.60:40693: received 0 of 4 requested bytes
I20260809 06:39:34.951138 9447 leader_election.cc:302] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13de037cec1e437089d863e57fc9b7e0; no voters: 01cf5ccc6e864fad86c2eda582b69553, 4c9d14f63b9445498f59d8cfbe5158ea
I20260809 06:39:34.951879 10051 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 06:39:36.148973 9355 leader_election.cc:334] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693): Timed out: RequestConsensusVote RPC to 127.6.111.60:40693 timed out after 2.143s (SENT)
I20260809 06:39:36.149199 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553; no voters: 13de037cec1e437089d863e57fc9b7e0, 4c9d14f63b9445498f59d8cfbe5158ea
I20260809 06:39:36.150024 10052 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 06:39:36.485831 10036 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:33.418197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.111.60:40693 (local address 127.0.0.1:38358)
0809 06:39:33.418418 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 06:39:33.418435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:39:33.418523 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:39:36.485628 (+3067105us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.6.111.60:40693: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 06:39:36.939764 10053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4c9d14f63b9445498f59d8cfbe5158ea)
I20260809 06:39:36.939967 10053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:36.940582 10053 leader_election.cc:288] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01cf5ccc6e864fad86c2eda582b69553 (127.6.111.62:39435), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:36.941249 9380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13de037cec1e437089d863e57fc9b7e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: false dest_uuid: "01cf5ccc6e864fad86c2eda582b69553" is_pre_election: true
I20260809 06:39:36.941632 9380 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13de037cec1e437089d863e57fc9b7e0 for term 4 because replica has last-logged OpId of term: 3 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20260809 06:39:38.396235 10055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4c9d14f63b9445498f59d8cfbe5158ea)
I20260809 06:39:38.396456 10055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:38.397161 10055 leader_election.cc:288] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:38.397738 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "13de037cec1e437089d863e57fc9b7e0" is_pre_election: true
I20260809 06:39:38.398005 9475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 3.
I20260809 06:39:38.398499 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0; no voters:
I20260809 06:39:38.398763 10055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 06:39:38.398900 10055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4c9d14f63b9445498f59d8cfbe5158ea)
I20260809 06:39:38.399012 10055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 3 FOLLOWER]: Advancing to term 4
I20260809 06:39:38.400959 10055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:38.401566 10055 leader_election.cc:288] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 election: Requested vote from peers 13de037cec1e437089d863e57fc9b7e0 (127.6.111.61:45875), 4c9d14f63b9445498f59d8cfbe5158ea (127.6.111.60:40693)
I20260809 06:39:38.402014 9475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "13de037cec1e437089d863e57fc9b7e0"
I20260809 06:39:38.402232 9475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 3 FOLLOWER]: Advancing to term 4
I20260809 06:39:38.404371 9475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01cf5ccc6e864fad86c2eda582b69553 in term 4.
I20260809 06:39:38.404798 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01cf5ccc6e864fad86c2eda582b69553, 13de037cec1e437089d863e57fc9b7e0; no voters:
I20260809 06:39:38.405114 10055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 06:39:38.405529 10055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [term 4 LEADER]: Becoming Leader. State: Replica: 01cf5ccc6e864fad86c2eda582b69553, State: Running, Role: LEADER
I20260809 06:39:38.405807 10055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 3.22, Last appended by leader: 22, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
I20260809 06:39:38.407040 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:38.407364 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:38.407929 10058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:38.409475 10058 catalog_manager.cc:669] Loaded metadata for table first_table [id=46993ea521294f0294cca2c23e4c27b8]
I20260809 06:39:38.409863 10058 catalog_manager.cc:669] Loaded metadata for table table-0 [id=91e6e5c7333e4d229498d98a023a507c]
I20260809 06:39:38.410171 10058 catalog_manager.cc:669] Loaded metadata for table second_table [id=dde222fb736a4beea0bf48fd5b9363f5]
I20260809 06:39:38.410482 10058 catalog_manager.cc:669] Loaded metadata for table table-1 [id=f83623fa273f450fbd3336ba464fd98a]
I20260809 06:39:38.411497 10058 tablet_loader.cc:96] loaded metadata for tablet 63886e36eac54288a8088e5b07b6de8d (table second_table [id=dde222fb736a4beea0bf48fd5b9363f5])
I20260809 06:39:38.411849 10058 tablet_loader.cc:96] loaded metadata for tablet 97d13e403e7545e19713960f526b7872 (table first_table [id=46993ea521294f0294cca2c23e4c27b8])
I20260809 06:39:38.412146 10058 tablet_loader.cc:96] loaded metadata for tablet d2eb6bd665ec4648809382e9bb656422 (table table-1 [id=f83623fa273f450fbd3336ba464fd98a])
I20260809 06:39:38.412432 10058 tablet_loader.cc:96] loaded metadata for tablet db9487eaa2c44c5b8683ad4f4e0924eb (table table-0 [id=91e6e5c7333e4d229498d98a023a507c])
I20260809 06:39:38.412757 10058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:38.413712 10058 catalog_manager.cc:1259] Loaded cluster ID: 6e3216e4e6da46248b0bee1f6a75f0eb
I20260809 06:39:38.413800 10058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:38.415004 10058 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:38.415807 10058 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553: Loaded TSK: 0
I20260809 06:39:38.416268 10058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:38.895478 9475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 4 FOLLOWER]: Refusing update from remote peer 01cf5ccc6e864fad86c2eda582b69553: Log matching property violated. Preceding OpId in replica: term: 3 index: 20. Preceding OpId from leader: term: 4 index: 23. (index mismatch)
I20260809 06:39:38.896248 10056 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 19, Time since last communication: 0.000s
I20260809 06:39:38.899776 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:38.900184 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:38.901677 10055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:38.901857 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:38.902045 10055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:38.902140 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 06:39:39.558010 10036 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:36.491170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.111.60:40693 (local address 127.0.0.1:38366)
0809 06:39:36.491391 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 06:39:36.491413 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:39:36.491539 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:39:39.557794 (+3066255us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.6.111.60:40693: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 06:39:39.595419 9447 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.6.111.60:40693 for RPC RequestConsensusVote timed out after 2.653s (ON_OUTBOUND_QUEUE)
I20260809 06:39:39.611312 9369 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50392:
name: "table-2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:39.632000 9723 tablet_service.cc:1467] Processing CreateTablet for tablet 995eb34df755480c83698c023a8c0957 (DEFAULT_TABLE table=table-2 [id=344fd3fb66a0407e8eeca29607882a67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:39.632470 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995eb34df755480c83698c023a8c0957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:39.636994 10095 tablet_bootstrap.cc:492] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e: Bootstrap starting.
I20260809 06:39:39.638531 10095 tablet_bootstrap.cc:654] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:39.640146 10095 tablet_bootstrap.cc:492] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e: No bootstrap required, opened a new log
I20260809 06:39:39.640254 10095 ts_tablet_manager.cc:1397] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:39.640877 10095 raft_consensus.cc:348] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:39.641040 10095 raft_consensus.cc:374] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:39.641111 10095 raft_consensus.cc:729] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Initialized, Role: FOLLOWER
I20260809 06:39:39.641407 10095 consensus_queue.cc:260] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:39.641613 10095 raft_consensus.cc:388] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:39.641712 10095 raft_consensus.cc:482] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:39.641824 10095 raft_consensus.cc:3037] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:39.643535 10095 raft_consensus.cc:504] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:39.643806 10095 leader_election.cc:302] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7fd5f5f0b64857b80983bfb25e219e; no voters:
I20260809 06:39:39.644255 10095 leader_election.cc:288] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:39.644397 10098 raft_consensus.cc:2781] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:39.645181 10095 ts_tablet_manager.cc:1428] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:39.645316 9810 heartbeater.cc:503] Master 127.6.111.62:39435 was elected leader, sending a full tablet report...
I20260809 06:39:39.645187 10098 raft_consensus.cc:686] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [term 1 LEADER]: Becoming Leader. State: Replica: 0e7fd5f5f0b64857b80983bfb25e219e, State: Running, Role: LEADER
I20260809 06:39:39.645869 10098 consensus_queue.cc:237] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } }
I20260809 06:39:39.650573 9368 catalog_manager.cc:5495] T 995eb34df755480c83698c023a8c0957 P 0e7fd5f5f0b64857b80983bfb25e219e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e7fd5f5f0b64857b80983bfb25e219e (127.6.111.1). New cstate: current_term: 1 leader_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7fd5f5f0b64857b80983bfb25e219e" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 42845 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:39.658113 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 9643
I20260809 06:39:39.658800 9571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea"
I20260809 06:39:39.659552 9574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" is_pre_election: true
I20260809 06:39:39.660053 9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" is_pre_election: true
I20260809 06:39:39.660198 9570 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 LEADER]: Stepping down as leader of term 3
I20260809 06:39:39.660311 9570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4c9d14f63b9445498f59d8cfbe5158ea, State: Running, Role: LEADER
I20260809 06:39:39.660743 9570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 3.22, Last appended by leader: 22, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } }
W20260809 06:39:39.661279 9838 consensus_queue.cc:1159] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "13de037cec1e437089d863e57fc9b7e0" responder_term: 3 status { last_received { term: 3 index: 20 } last_committed_idx: 19 last_received_current_leader { term: 3 index: 20 } }
I20260809 06:39:39.661584 9573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01cf5ccc6e864fad86c2eda582b69553" candidate_term: 4 candidate_status { last_received { term: 3 index: 22 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" is_pre_election: true
I20260809 06:39:39.664570 9570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 3 FOLLOWER]: Advancing to term 4
W20260809 06:39:39.666253 10104 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:39.662321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.6.111.60:40693)
0809 06:39:39.664842 (+ 2521us) server_negotiation.cc:206] Beginning negotiation
0809 06:39:39.664850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:39:39.664870 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:39:39.664926 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:39:39.664931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:39:39.664999 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:39:39.665241 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54570: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2405,"thread_start_us":2311,"threads_started":1}
W20260809 06:39:39.667181 10104 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:39.664370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.6.111.60:40693)
0809 06:39:39.666469 (+ 2099us) server_negotiation.cc:206] Beginning negotiation
0809 06:39:39.666474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:39:39.666491 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:39:39.666572 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:39:39.666578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:39:39.666667 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:39:39.666864 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54576: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2002,"threads_started":1}
W20260809 06:39:39.667883 10104 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:39.664776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.6.111.60:40693)
0809 06:39:39.667336 (+ 2560us) server_negotiation.cc:206] Beginning negotiation
0809 06:39:39.667341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:39:39.667357 (+ 16us) server_negotiation.cc:407] Connection header received
0809 06:39:39.667415 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:39:39.667420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:39:39.667474 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:39:39.667647 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38358: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":2481,"threads_started":1}
I20260809 06:39:39.667995 9570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [term 4 FOLLOWER]: Refusing update from remote peer 01cf5ccc6e864fad86c2eda582b69553: Log matching property violated. Preceding OpId in replica: term: 3 index: 22. Preceding OpId from leader: term: 4 index: 23. (index mismatch)
I20260809 06:39:39.674463 10089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01cf5ccc6e864fad86c2eda582b69553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260809 06:39:39.675076 10107 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:39:39.673695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.6.111.60:40693)
0809 06:39:39.674161 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 06:39:39.674169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:39:39.674190 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:39:39.674246 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:39:39.674252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:39:39.674345 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:39:39.674717 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38366: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":323}
I20260809 06:39:39.683023 9573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13de037cec1e437089d863e57fc9b7e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: false dest_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" is_pre_election: true
I20260809 06:39:39.683710 9447 leader_election.cc:302] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13de037cec1e437089d863e57fc9b7e0; no voters: 01cf5ccc6e864fad86c2eda582b69553, 4c9d14f63b9445498f59d8cfbe5158ea
I20260809 06:39:39.696002 10113 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 13de037cec1e437089d863e57fc9b7e0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 06:39:39.700301 9802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:39.705304 9839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:39.705626 9839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:39.706423 9838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01cf5ccc6e864fad86c2eda582b69553. Latest consensus state: current_term: 4 leader_uuid: "01cf5ccc6e864fad86c2eda582b69553" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "01cf5ccc6e864fad86c2eda582b69553" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 39435 } } peers { permanent_uuid: "13de037cec1e437089d863e57fc9b7e0" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 45875 } } peers { permanent_uuid: "4c9d14f63b9445498f59d8cfbe5158ea" member_type: VOTER last_known_addr { host: "127.6.111.60" port: 40693 } attrs { promote: false } } }
I20260809 06:39:39.706717 9838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c9d14f63b9445498f59d8cfbe5158ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:39.708690 10108 mvcc.cc:205] Tried to move back new op lower bound from 7316511046253797376 to 7316511041152315392. Current Snapshot: MvccSnapshot[applied={T|T < 7316511046253797376}]
I20260809 06:39:39.821655 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 9643
I20260809 06:39:39.854887 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 9333
I20260809 06:39:39.855942 9402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:39.973064 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 9333
I20260809 06:39:40.004261 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 9409
I20260809 06:39:40.005432 9499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:40.117991 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 9409
I20260809 06:39:40.149300 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 9508
I20260809 06:39:40.150417 9598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:40.266350 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 9508
2026-08-09T06:39:40Z chronyd exiting
[ OK ] ParameterizedAddMasterTest.TestAddMasterCatchupFromWAL/2 (20371 ms)
[----------] 1 test from ParameterizedAddMasterTest (20371 ms total)
[----------] 1 test from ParameterizedRemoveMasterTest
[ RUN ] ParameterizedRemoveMasterTest.TestRemoveMaster/0
2026-08-09T06:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:40Z Disabled control of system clock
I20260809 06:39:40.350107 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:42253
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:42253,127.6.111.61:35589
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260809 06:39:40.694664 10126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:40.694977 10126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:40.695055 10126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:40.702067 10126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:40.702162 10126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:40.702204 10126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:40.702261 10126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:40.711161 10126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:42253,127.6.111.61:35589
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:42253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:40.711450 10126 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:42253,127.6.111.61:35589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:40.712515 10126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:40.714257 10126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:40.727094 10132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:40.727068 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:40.727099 10135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:40.727689 10126 server_base.cc:1034] running on GCE node
I20260809 06:39:40.728572 10126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:40.730044 10126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:40.731278 10126 hybrid_clock.cc:648] HybridClock initialized: now 1786257580731237 us; error 56 us; skew 500 ppm
I20260809 06:39:40.731635 10126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:40.733943 10126 webserver.cc:460] Webserver started at http://127.6.111.62:37279/ using document root <none> and password file <none>
I20260809 06:39:40.734650 10126 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:40.734797 10126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:40.735134 10126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:40.737857 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "95a38be114064bc3b8991db1ebfb1ae3"
format_stamp: "Formatted at 2026-08-09 06:39:40 on dist-test-slave-bw88"
I20260809 06:39:40.738655 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "95a38be114064bc3b8991db1ebfb1ae3"
format_stamp: "Formatted at 2026-08-09 06:39:40 on dist-test-slave-bw88"
I20260809 06:39:40.743712 10126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:40.746992 10142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:40.748200 10126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 06:39:40.748378 10126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "95a38be114064bc3b8991db1ebfb1ae3"
format_stamp: "Formatted at 2026-08-09 06:39:40 on dist-test-slave-bw88"
I20260809 06:39:40.748610 10126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:40.760269 10126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:40.761030 10126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:40.761432 10126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:40.779914 10126 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:42253
I20260809 06:39:40.779939 10207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:42253 every 8 connection(s)
I20260809 06:39:40.781217 10126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:40.784858 10209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:40.790948 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10126
I20260809 06:39:40.791201 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:40.791124 10209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:40.792739 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:35589
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:42253,127.6.111.61:35589
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260809 06:39:40.809993 10209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:39:40.811308 10146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
W20260809 06:39:40.813058 10209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35589: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
I20260809 06:39:40.862399 10209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } attempt: 1
W20260809 06:39:40.864565 10209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35589: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
I20260809 06:39:40.932907 10209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } attempt: 2
W20260809 06:39:40.935057 10209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35589: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
I20260809 06:39:41.026427 10209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } attempt: 3
W20260809 06:39:41.028537 10209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35589: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
W20260809 06:39:41.140175 10214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:41.140573 10214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:41.140730 10214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:41.147962 10214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:41.148123 10214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:41.148228 10214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:41.148335 10214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:41.157935 10214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:42253,127.6.111.61:35589
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:35589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:41.158355 10214 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:42253,127.6.111.61:35589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:41.159526 10214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:41.161441 10214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 06:39:41.171774 10209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } attempt: 4
W20260809 06:39:41.174607 10209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35589: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
W20260809 06:39:41.174711 10223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:41.174798 10222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:41.174767 10225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:41.176831 10214 server_base.cc:1034] running on GCE node
I20260809 06:39:41.177677 10214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:41.179175 10214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:41.180410 10214 hybrid_clock.cc:648] HybridClock initialized: now 1786257581180342 us; error 61 us; skew 500 ppm
I20260809 06:39:41.180768 10214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:41.183216 10214 webserver.cc:460] Webserver started at http://127.6.111.61:45915/ using document root <none> and password file <none>
I20260809 06:39:41.183962 10214 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:41.184096 10214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:41.184434 10214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:41.187319 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "81674e542b6a441a85bea2e02361e8db"
format_stamp: "Formatted at 2026-08-09 06:39:41 on dist-test-slave-bw88"
I20260809 06:39:41.188153 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "81674e542b6a441a85bea2e02361e8db"
format_stamp: "Formatted at 2026-08-09 06:39:41 on dist-test-slave-bw88"
I20260809 06:39:41.193619 10214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 06:39:41.197082 10231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:41.198455 10214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:41.198662 10214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "81674e542b6a441a85bea2e02361e8db"
format_stamp: "Formatted at 2026-08-09 06:39:41 on dist-test-slave-bw88"
I20260809 06:39:41.198901 10214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:41.218890 10214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:41.219723 10214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:41.220031 10214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:41.238564 10214 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:35589
I20260809 06:39:41.238595 10294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:35589 every 8 connection(s)
I20260809 06:39:41.239884 10214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:41.243680 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:41.250250 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10214
I20260809 06:39:41.250430 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:41.249717 10295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:41.273419 10295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:41.283442 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: Bootstrap starting.
I20260809 06:39:41.287150 10295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:41.288146 10295 log.cc:824] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:41.290848 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: No bootstrap required, opened a new log
I20260809 06:39:41.297936 10295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.298352 10295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:41.298485 10295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81674e542b6a441a85bea2e02361e8db, State: Initialized, Role: FOLLOWER
I20260809 06:39:41.299063 10295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.299911 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.300189 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:41.300884 10295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:41.309583 10320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:41.309729 10320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 06:39:41.472121 10238 tablet.cc:2369] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:41.473897 10209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } attempt: 5
I20260809 06:39:41.484597 10209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Bootstrap starting.
I20260809 06:39:41.487850 10209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:41.488965 10209 log.cc:824] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:41.492158 10209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: No bootstrap required, opened a new log
I20260809 06:39:41.503176 10209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.503577 10209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:41.503736 10209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a38be114064bc3b8991db1ebfb1ae3, State: Initialized, Role: FOLLOWER
I20260809 06:39:41.504515 10209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.505493 10324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.505862 10324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:41.506897 10209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:41.514094 10149 tablet.cc:2369] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:39:41.516804 10336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:41.516938 10336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:41.519150 10305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:41.519436 10305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.520862 10305 leader_election.cc:288] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95a38be114064bc3b8991db1ebfb1ae3 (127.6.111.62:42253)
I20260809 06:39:41.521842 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81674e542b6a441a85bea2e02361e8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a38be114064bc3b8991db1ebfb1ae3" is_pre_election: true
I20260809 06:39:41.522475 10174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81674e542b6a441a85bea2e02361e8db in term 0.
I20260809 06:39:41.523193 10236 leader_election.cc:302] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 81674e542b6a441a85bea2e02361e8db, 95a38be114064bc3b8991db1ebfb1ae3; no voters:
I20260809 06:39:41.523648 10305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:39:41.523808 10305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:41.523931 10305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:41.527333 10305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.527891 10305 leader_election.cc:288] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [CANDIDATE]: Term 1 election: Requested vote from peers 95a38be114064bc3b8991db1ebfb1ae3 (127.6.111.62:42253)
I20260809 06:39:41.528478 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81674e542b6a441a85bea2e02361e8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a38be114064bc3b8991db1ebfb1ae3"
I20260809 06:39:41.528712 10174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:41.531961 10174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81674e542b6a441a85bea2e02361e8db in term 1.
I20260809 06:39:41.532377 10236 leader_election.cc:302] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 81674e542b6a441a85bea2e02361e8db, 95a38be114064bc3b8991db1ebfb1ae3; no voters:
I20260809 06:39:41.532756 10305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:41.533455 10305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 LEADER]: Becoming Leader. State: Replica: 81674e542b6a441a85bea2e02361e8db, State: Running, Role: LEADER
I20260809 06:39:41.534057 10305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:41.538025 10339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81674e542b6a441a85bea2e02361e8db. Latest consensus state: current_term: 1 leader_uuid: "81674e542b6a441a85bea2e02361e8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.538398 10339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:41.540417 10341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:41.542755 10341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:41.551697 10174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 1 FOLLOWER]: Refusing update from remote peer 81674e542b6a441a85bea2e02361e8db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:39:41.552757 10339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:41.561486 10324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81674e542b6a441a85bea2e02361e8db. Latest consensus state: current_term: 1 leader_uuid: "81674e542b6a441a85bea2e02361e8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.561724 10324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:41.565272 10342 mvcc.cc:205] Tried to move back new op lower bound from 7316511054024765440 to 7316511053968064512. Current Snapshot: MvccSnapshot[applied={T|T < 7316511054024765440}]
I20260809 06:39:41.573786 10339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81674e542b6a441a85bea2e02361e8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.574034 10339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:41.576190 10341 catalog_manager.cc:1347] Generated new cluster ID: 2cc1b247f12e459a927b16e5b5db42d7
I20260809 06:39:41.576293 10341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:41.581897 10324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81674e542b6a441a85bea2e02361e8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:41.582136 10324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:41.593601 10341 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:41.595463 10341 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:41.607911 10341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: Generated new TSK 0
I20260809 06:39:41.608974 10341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:41.619151 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:42253,127.6.111.61:35589
--builtin_ntp_servers=127.6.111.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:42.005401 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:42.005671 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:42.005756 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:42.013444 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:42.013657 10348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:42.022508 10348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:42253,127.6.111.61:35589
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:42.023816 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:42.025722 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:42.038614 10353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:42.038789 10355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:42.039166 10357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:42.039801 10348 server_base.cc:1034] running on GCE node
I20260809 06:39:42.040437 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:42.041414 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:42.042660 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786257582042593 us; error 40 us; skew 500 ppm
I20260809 06:39:42.042994 10348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:42.045259 10348 webserver.cc:460] Webserver started at http://127.6.111.1:39811/ using document root <none> and password file <none>
I20260809 06:39:42.045964 10348 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:42.046126 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:42.046434 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:42.049016 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "85a833d4527f46079cde0b3a4f974334"
format_stamp: "Formatted at 2026-08-09 06:39:42 on dist-test-slave-bw88"
I20260809 06:39:42.049821 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "85a833d4527f46079cde0b3a4f974334"
format_stamp: "Formatted at 2026-08-09 06:39:42 on dist-test-slave-bw88"
I20260809 06:39:42.054970 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:42.058313 10366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:42.059552 10348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:39:42.059726 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "85a833d4527f46079cde0b3a4f974334"
format_stamp: "Formatted at 2026-08-09 06:39:42 on dist-test-slave-bw88"
I20260809 06:39:42.059931 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:42.074085 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:42.074872 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:42.075196 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:42.076153 10348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:42.077756 10348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:42.077862 10348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:42.078034 10348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:42.078115 10348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:42.114460 10348 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:45693
I20260809 06:39:42.114500 10518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:45693 every 8 connection(s)
I20260809 06:39:42.115850 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:42.117956 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10348
I20260809 06:39:42.118228 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:42.139273 10519 heartbeater.cc:347] Connected to a master server at 127.6.111.61:35589
I20260809 06:39:42.139628 10519 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:42.140393 10519 heartbeater.cc:511] Master 127.6.111.61:35589 requested a full tablet report, sending...
I20260809 06:39:42.141187 10520 heartbeater.cc:347] Connected to a master server at 127.6.111.62:42253
I20260809 06:39:42.141323 10520 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:42.141918 10520 heartbeater.cc:511] Master 127.6.111.62:42253 requested a full tablet report, sending...
I20260809 06:39:42.142779 10250 ts_manager.cc:194] Registered new tserver with Master: 85a833d4527f46079cde0b3a4f974334 (127.6.111.1:45693)
I20260809 06:39:42.143777 10160 ts_manager.cc:194] Registered new tserver with Master: 85a833d4527f46079cde0b3a4f974334 (127.6.111.1:45693)
I20260809 06:39:42.144752 10250 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:38287
I20260809 06:39:42.153402 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:42.158449 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:42.166746 6588 dynamic_multi_master-test.cc:972] Transferring leadership to master: 95a38be114064bc3b8991db1ebfb1ae3
I20260809 06:39:42.169209 10262 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "81674e542b6a441a85bea2e02361e8db"
mode: GRACEFUL
new_leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3"
from {username='slave'} at 127.0.0.1:42662
I20260809 06:39:42.169576 10262 raft_consensus.cc:595] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 LEADER]: Received request to transfer leadership to TS 95a38be114064bc3b8991db1ebfb1ae3
I20260809 06:39:42.377203 10305 raft_consensus.cc:979] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db: : Instructing follower 95a38be114064bc3b8991db1ebfb1ae3 to start an election
I20260809 06:39:42.377429 10305 raft_consensus.cc:1067] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 LEADER]: Signalling peer 95a38be114064bc3b8991db1ebfb1ae3 to start an election
I20260809 06:39:42.378351 10174 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "95a38be114064bc3b8991db1ebfb1ae3"
from {username='slave'} at 127.0.0.1:50918
I20260809 06:39:42.378640 10174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 06:39:42.378796 10174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:42.380787 10174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:42.382001 10174 leader_election.cc:288] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [CANDIDATE]: Term 2 election: Requested vote from peers 81674e542b6a441a85bea2e02361e8db (127.6.111.61:35589)
I20260809 06:39:42.382684 10262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95a38be114064bc3b8991db1ebfb1ae3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "81674e542b6a441a85bea2e02361e8db"
I20260809 06:39:42.382941 10262 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 LEADER]: Stepping down as leader of term 1
I20260809 06:39:42.383021 10262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 81674e542b6a441a85bea2e02361e8db, State: Running, Role: LEADER
I20260809 06:39:42.383291 10262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:42.383698 10262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:42.385811 10262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a38be114064bc3b8991db1ebfb1ae3 in term 2.
I20260809 06:39:42.386246 10146 leader_election.cc:302] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 81674e542b6a441a85bea2e02361e8db, 95a38be114064bc3b8991db1ebfb1ae3; no voters:
I20260809 06:39:42.386907 10532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:39:42.387300 10532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 LEADER]: Becoming Leader. State: Replica: 95a38be114064bc3b8991db1ebfb1ae3, State: Running, Role: LEADER
I20260809 06:39:42.387706 10532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } }
I20260809 06:39:42.388876 10534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a38be114064bc3b8991db1ebfb1ae3. Latest consensus state: current_term: 2 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:42.389176 10534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:42.389804 10535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:42.391911 10535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:42.392786 10535 catalog_manager.cc:1259] Loaded cluster ID: 2cc1b247f12e459a927b16e5b5db42d7
I20260809 06:39:42.392859 10535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:42.395429 10535 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:42.396414 10535 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Loaded TSK: 0
I20260809 06:39:42.397773 10535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:42.460726 10160 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48746:
name: "first_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:42.472733 10262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [term 2 FOLLOWER]: Refusing update from remote peer 95a38be114064bc3b8991db1ebfb1ae3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 06:39:42.473536 10534 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 06:39:42.476255 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a38be114064bc3b8991db1ebfb1ae3. Latest consensus state: current_term: 2 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:42.476516 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:42.478000 10532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:42.478262 10532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:42.478829 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } peers { permanent_uuid: "81674e542b6a441a85bea2e02361e8db" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35589 } } }
I20260809 06:39:42.479122 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81674e542b6a441a85bea2e02361e8db [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:42.497172 10428 tablet_service.cc:1467] Processing CreateTablet for tablet 02730bfc2ec84478acd8ca4d8fca27ec (DEFAULT_TABLE table=first_table [id=4b4313063be54c7f9d40942fd6b1bd29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:42.498588 10428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02730bfc2ec84478acd8ca4d8fca27ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:42.510113 10553 tablet_bootstrap.cc:492] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Bootstrap starting.
I20260809 06:39:42.513098 10553 tablet_bootstrap.cc:654] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:42.514207 10553 log.cc:824] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:42.516832 10553 tablet_bootstrap.cc:492] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: No bootstrap required, opened a new log
I20260809 06:39:42.517107 10553 ts_tablet_manager.cc:1397] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 06:39:42.524462 10553 raft_consensus.cc:348] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:42.524803 10553 raft_consensus.cc:374] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:42.524915 10553 raft_consensus.cc:729] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a833d4527f46079cde0b3a4f974334, State: Initialized, Role: FOLLOWER
I20260809 06:39:42.525507 10553 consensus_queue.cc:260] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:42.525784 10553 raft_consensus.cc:388] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:42.525934 10553 raft_consensus.cc:482] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:42.526124 10553 raft_consensus.cc:3037] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:42.528069 10553 raft_consensus.cc:504] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:42.528571 10553 leader_election.cc:302] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85a833d4527f46079cde0b3a4f974334; no voters:
I20260809 06:39:42.529162 10553 leader_election.cc:288] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:42.529299 10555 raft_consensus.cc:2781] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:42.530066 10555 raft_consensus.cc:686] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 LEADER]: Becoming Leader. State: Replica: 85a833d4527f46079cde0b3a4f974334, State: Running, Role: LEADER
I20260809 06:39:42.530447 10519 heartbeater.cc:503] Master 127.6.111.61:35589 was elected leader, sending a full tablet report...
I20260809 06:39:42.531088 10553 ts_tablet_manager.cc:1428] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Time spent starting tablet: real 0.014s user 0.005s sys 0.010s
I20260809 06:39:42.530983 10555 consensus_queue.cc:237] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:42.536800 10160 catalog_manager.cc:5495] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85a833d4527f46079cde0b3a4f974334 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "85a833d4527f46079cde0b3a4f974334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:42.543262 10520 heartbeater.cc:503] Master 127.6.111.62:42253 was elected leader, sending a full tablet report...
W20260809 06:39:42.897887 10561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:42.898267 10561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:42.905568 10561 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20260809 06:39:42.943974 10160 catalog_manager.cc:6979] Initiating ChangeConfig request to remove master 127.6.111.61:35589: tablet_id: "00000000000000000000000000000000"
type: REMOVE_PEER
server {
permanent_uuid: "81674e542b6a441a85bea2e02361e8db"
}
dest_uuid: "95a38be114064bc3b8991db1ebfb1ae3"
cas_config_opid_index: -1
I20260809 06:39:42.944902 10160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } }
I20260809 06:39:42.945850 10532 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index -1 to 9, VOTER 81674e542b6a441a85bea2e02361e8db (127.6.111.61) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } }
I20260809 06:39:42.948251 10532 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to remove master 127.6.111.61:35589
I20260809 06:39:42.948679 10534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } }
I20260809 06:39:42.948926 10534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:42.949009 10561 tool_action_master.cc:566] Successfully removed master 127.6.111.61:35589 from the cluster. Please follow the next steps from the Kudu administration documentation on "Removing Kudu Masters from a Multi-Master Deployment" or "Recovering from a dead Kudu Master in a Multi-Master Deployment" as appropriate.
W20260809 06:39:43.034557 6588 subprocess.cc:277] Child process 10214 (/tmp/dist-test-task_4zeaO/build/asan/bin/kudu --fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/wal --fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data --block_manager=log --webserver_interface=localhost --never_fsync --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/logs --server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --logtostderr --logbuflevel=-1 --ipki_server_key_size=768 --openssl_security_level_override=0 master run --ipki_ca_key_size=768 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.6.111.61:35589 --webserver_interface=127.6.111.61 --webserver_port=0 --builtin_ntp_servers=127.6.111.20:36385 --builtin_ntp_poll_interval_ms=100 --ntp_initial_sync_wait_secs=10 --time_source=builtin --rpc_reuseport=true --master_addresses=127.6.111.62:42253,127.6.111.61:35589 --consensus_rpc_timeout_ms=2000 --follower_unavailable_considered_failed_sec=4 --master_auto_join_cluster=false) was orphaned. Sending signal 9...
I20260809 06:39:43.152931 10428 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
95a38be114064bc3b8991db1ebfb1ae3 | 127.6.111.62:42253 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36385 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
85a833d4527f46079cde0b3a4f974334 | 127.6.111.1:45693 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.6.111.1 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:36385 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260809 06:39:43.183602 6588 log_verifier.cc:126] Checking tablet 02730bfc2ec84478acd8ca4d8fca27ec
I20260809 06:39:43.192619 6588 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 02730bfc2ec84478acd8ca4d8fca27ec
W20260809 06:39:43.541270 10519 heartbeater.cc:644] Failed to heartbeat to 127.6.111.61:35589 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.111.61:35589: connect: Connection refused (error 111)
E20260809 06:39:43.623226 10160 master_service.cc:344] Failed removing master 127.6.111.61:35589. Not found: Master 127.6.111.61:35589 not found
I20260809 06:39:43.710194 6588 dynamic_multi_master-test.cc:1021] Transferring leadership to master: 81674e542b6a441a85bea2e02361e8db
I20260809 06:39:43.713572 10174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "95a38be114064bc3b8991db1ebfb1ae3"
mode: GRACEFUL
new_leader_uuid: "81674e542b6a441a85bea2e02361e8db"
from {username='slave'} at 127.0.0.1:50916
I20260809 06:39:43.713842 10174 raft_consensus.cc:595] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 LEADER]: Received request to transfer leadership to TS 81674e542b6a441a85bea2e02361e8db
I20260809 06:39:43.713985 10174 raft_consensus.cc:619] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 LEADER]: Rejecting request to transfer leadership because tablet server 81674e542b6a441a85bea2e02361e8db is not a voter in the active config
I20260809 06:39:43.714985 6588 dynamic_multi_master-test.cc:1028] Shutting down the old cluster
I20260809 06:39:43.715133 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 10348
I20260809 06:39:43.721715 10514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:43.832259 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 10348
I20260809 06:39:43.863982 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 10126
I20260809 06:39:43.865155 10200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:43.975255 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 10126
2026-08-09T06:39:44Z chronyd exiting
I20260809 06:39:44.018723 6588 dynamic_multi_master-test.cc:1031] Bringing up the migrated cluster
2026-08-09T06:39:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:44Z Disabled control of system clock
I20260809 06:39:44.053784 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:42253
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:42253
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260809 06:39:44.396080 10628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:44.396361 10628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:44.396456 10628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:44.403601 10628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:44.403735 10628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:44.403815 10628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:44.403921 10628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:39:44.412585 10628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:42253
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:42253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:44.413972 10628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:44.416237 10628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:44.428602 10633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:44.428599 10634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:44.428586 10637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:44.429280 10628 server_base.cc:1034] running on GCE node
I20260809 06:39:44.430083 10628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:44.431453 10628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:44.432695 10628 hybrid_clock.cc:648] HybridClock initialized: now 1786257584432627 us; error 41 us; skew 500 ppm
I20260809 06:39:44.433040 10628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:44.435325 10628 webserver.cc:460] Webserver started at http://127.6.111.62:43651/ using document root <none> and password file <none>
I20260809 06:39:44.436015 10628 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:44.436182 10628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:44.441386 10628 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:44.444275 10643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:44.445643 10628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:44.445823 10628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "95a38be114064bc3b8991db1ebfb1ae3"
format_stamp: "Formatted at 2026-08-09 06:39:40 on dist-test-slave-bw88"
I20260809 06:39:44.446789 10628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:44.465981 10628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:44.466851 10628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:44.467157 10628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:44.486980 10628 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:42253
I20260809 06:39:44.487028 10710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:42253 every 8 connection(s)
I20260809 06:39:44.488623 10628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:44.493672 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10628
I20260809 06:39:44.493906 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:44.496244 10711 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:39:44.500243 10711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Bootstrap starting.
I20260809 06:39:44.519164 10711 log.cc:824] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:44.530915 10711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:44.531458 10711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Bootstrap complete.
I20260809 06:39:44.540143 10711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } }
I20260809 06:39:44.541050 10711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a38be114064bc3b8991db1ebfb1ae3, State: Initialized, Role: FOLLOWER
I20260809 06:39:44.541695 10711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } }
I20260809 06:39:44.541958 10711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:44.542102 10711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:44.542301 10711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:39:44.546442 10711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } }
I20260809 06:39:44.547000 10711 leader_election.cc:302] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95a38be114064bc3b8991db1ebfb1ae3; no voters:
I20260809 06:39:44.547607 10711 leader_election.cc:288] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 06:39:44.547772 10716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 06:39:44.548579 10716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [term 3 LEADER]: Becoming Leader. State: Replica: 95a38be114064bc3b8991db1ebfb1ae3, State: Running, Role: LEADER
I20260809 06:39:44.549106 10716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } }
I20260809 06:39:44.549889 10711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:44.551949 10718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a38be114064bc3b8991db1ebfb1ae3. Latest consensus state: current_term: 3 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } }
I20260809 06:39:44.551921 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "95a38be114064bc3b8991db1ebfb1ae3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "95a38be114064bc3b8991db1ebfb1ae3" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 42253 } } }
I20260809 06:39:44.552330 10718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:44.552333 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:44.553479 10722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:44.559419 10722 catalog_manager.cc:669] Loaded metadata for table first_table [id=4b4313063be54c7f9d40942fd6b1bd29]
I20260809 06:39:44.562024 10722 tablet_loader.cc:96] loaded metadata for tablet 02730bfc2ec84478acd8ca4d8fca27ec (table first_table [id=4b4313063be54c7f9d40942fd6b1bd29])
I20260809 06:39:44.562384 10722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:44.563701 10722 catalog_manager.cc:1259] Loaded cluster ID: 2cc1b247f12e459a927b16e5b5db42d7
I20260809 06:39:44.563797 10722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:44.567711 10722 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:44.569178 10722 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95a38be114064bc3b8991db1ebfb1ae3: Loaded TSK: 0
I20260809 06:39:44.570886 10722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:44.583667 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:42253
--builtin_ntp_servers=127.6.111.20:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 06:39:44.942320 10738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:44.942595 10738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:44.942770 10738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:44.949932 10738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:44.950181 10738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:44.959528 10738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:42253
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:44.960852 10738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:44.962774 10738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:44.975907 10746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:44.976017 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:44.976047 10749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:44.976891 10738 server_base.cc:1034] running on GCE node
I20260809 06:39:44.977569 10738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:44.978490 10738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:44.979740 10738 hybrid_clock.cc:648] HybridClock initialized: now 1786257584979659 us; error 46 us; skew 500 ppm
I20260809 06:39:44.980077 10738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:44.982416 10738 webserver.cc:460] Webserver started at http://127.6.111.1:34057/ using document root <none> and password file <none>
I20260809 06:39:44.983096 10738 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:44.983209 10738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:44.988363 10738 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 06:39:44.991405 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:44.992714 10738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:44.992887 10738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "85a833d4527f46079cde0b3a4f974334"
format_stamp: "Formatted at 2026-08-09 06:39:42 on dist-test-slave-bw88"
I20260809 06:39:44.993948 10738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:45.014990 10738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:45.015853 10738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:45.016149 10738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:45.017155 10738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:45.019160 10765 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 06:39:45.024631 10738 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 06:39:45.024768 10738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 06:39:45.024906 10738 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 06:39:45.027920 10738 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 06:39:45.028023 10738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260809 06:39:45.028379 10765 tablet_bootstrap.cc:492] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Bootstrap starting.
I20260809 06:39:45.048010 10765 log.cc:824] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:45.054541 10765 tablet_bootstrap.cc:492] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:39:45.055348 10765 tablet_bootstrap.cc:492] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Bootstrap complete.
I20260809 06:39:45.057459 10765 ts_tablet_manager.cc:1397] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.004s
I20260809 06:39:45.068763 10738 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:41803
I20260809 06:39:45.068890 10906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:41803 every 8 connection(s)
I20260809 06:39:45.070333 10738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:45.071663 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10738
I20260809 06:39:45.071875 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:45.072090 10765 raft_consensus.cc:348] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:45.073275 10765 raft_consensus.cc:729] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a833d4527f46079cde0b3a4f974334, State: Initialized, Role: FOLLOWER
I20260809 06:39:45.074116 10765 consensus_queue.cc:260] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:45.074396 10765 raft_consensus.cc:388] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:45.074568 10765 raft_consensus.cc:482] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:45.074811 10765 raft_consensus.cc:3037] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:39:45.079823 10765 raft_consensus.cc:504] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:45.080354 10765 leader_election.cc:302] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85a833d4527f46079cde0b3a4f974334; no voters:
I20260809 06:39:45.081013 10765 leader_election.cc:288] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:39:45.081246 10913 raft_consensus.cc:2781] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:39:45.082659 10913 raft_consensus.cc:686] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [term 2 LEADER]: Becoming Leader. State: Replica: 85a833d4527f46079cde0b3a4f974334, State: Running, Role: LEADER
I20260809 06:39:45.082849 10765 ts_tablet_manager.cc:1428] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334: Time spent starting tablet: real 0.025s user 0.019s sys 0.005s
I20260809 06:39:45.083261 10913 consensus_queue.cc:237] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } }
I20260809 06:39:45.095400 10909 heartbeater.cc:347] Connected to a master server at 127.6.111.62:42253
I20260809 06:39:45.095768 10909 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:45.096720 10909 heartbeater.cc:511] Master 127.6.111.62:42253 requested a full tablet report, sending...
I20260809 06:39:45.100704 10665 ts_manager.cc:194] Registered new tserver with Master: 85a833d4527f46079cde0b3a4f974334 (127.6.111.1:41803)
I20260809 06:39:45.101997 10665 catalog_manager.cc:5495] T 02730bfc2ec84478acd8ca4d8fca27ec P 85a833d4527f46079cde0b3a4f974334 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "85a833d4527f46079cde0b3a4f974334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a833d4527f46079cde0b3a4f974334" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 45693 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:45.113093 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:45.114626 10665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:58767
I20260809 06:39:45.117517 10909 heartbeater.cc:503] Master 127.6.111.62:42253 was elected leader, sending a full tablet report...
I20260809 06:39:45.202715 10820 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
95a38be114064bc3b8991db1ebfb1ae3 | 127.6.111.62:42253 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:39235 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
85a833d4527f46079cde0b3a4f974334 | 127.6.111.1:41803 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 1
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
local_ip_for_outbound_sockets | 127.6.111.1 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRemoveMasterTest.TestRemoveMaster_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb | hidden | all 1 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.6.111.20:39235 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 2 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
first_table | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 1
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260809 06:39:45.233532 6588 log_verifier.cc:126] Checking tablet 02730bfc2ec84478acd8ca4d8fca27ec
I20260809 06:39:45.242316 6588 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 02730bfc2ec84478acd8ca4d8fca27ec
I20260809 06:39:45.272949 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 10738
I20260809 06:39:45.280781 10898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:45.373023 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 10738
I20260809 06:39:45.400906 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 10628
I20260809 06:39:45.402158 10705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:39:45.510602 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 10628
2026-08-09T06:39:45Z chronyd exiting
[ OK ] ParameterizedRemoveMasterTest.TestRemoveMaster/0 (5245 ms)
[----------] 1 test from ParameterizedRemoveMasterTest (5245 ms total)
[----------] 1 test from ParameterizedRecoverMasterTest
[ RUN ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/0
2026-08-09T06:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:39:45Z Disabled control of system clock
I20260809 06:39:45.597514 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:38043
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:38043,127.6.111.61:35251
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
W20260809 06:39:45.930092 10978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:45.930395 10978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:45.930495 10978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:45.938220 10978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:45.938333 10978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:45.938376 10978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:45.938434 10978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:39:45.938503 10978 flags.cc:425] Enabled experimental flag: --flush_threshold_secs=0
I20260809 06:39:45.947944 10978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:38043,127.6.111.61:35251
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:38043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:45.948269 10978 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:38043,127.6.111.61:35251'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:45.949458 10978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:45.951308 10978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:45.963174 10984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:45.963152 10987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:45.963207 10985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:45.964293 10978 server_base.cc:1034] running on GCE node
I20260809 06:39:45.965070 10978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:45.966500 10978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:45.967721 10978 hybrid_clock.cc:648] HybridClock initialized: now 1786257585967662 us; error 29 us; skew 500 ppm
I20260809 06:39:45.968067 10978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:45.970438 10978 webserver.cc:460] Webserver started at http://127.6.111.62:45131/ using document root <none> and password file <none>
I20260809 06:39:45.971127 10978 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:45.971267 10978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:45.971585 10978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:45.974464 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "5d3ee6fced0d4ee2ba409f58ea250645"
format_stamp: "Formatted at 2026-08-09 06:39:45 on dist-test-slave-bw88"
I20260809 06:39:45.975278 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "5d3ee6fced0d4ee2ba409f58ea250645"
format_stamp: "Formatted at 2026-08-09 06:39:45 on dist-test-slave-bw88"
I20260809 06:39:45.980306 10978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 06:39:45.983745 10998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:45.985033 10978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:39:45.985239 10978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "5d3ee6fced0d4ee2ba409f58ea250645"
format_stamp: "Formatted at 2026-08-09 06:39:45 on dist-test-slave-bw88"
I20260809 06:39:45.985517 10978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:46.007485 10978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:46.008379 10978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:46.008754 10978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:46.027750 10978 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:38043
I20260809 06:39:46.027773 11064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:38043 every 8 connection(s)
I20260809 06:39:46.029138 10978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:39:46.032931 11065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:46.037823 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 10978
I20260809 06:39:46.038055 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:39:46.039341 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:35251
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:38043,127.6.111.61:35251
--flush_threshold_secs=0
--log_segment_size_mb=1
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--master_auto_join_cluster=false with env {}
I20260809 06:39:46.039162 11065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:46.059602 11065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:39:46.060770 11002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
W20260809 06:39:46.062454 11065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35251: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
I20260809 06:39:46.111912 11065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } attempt: 1
W20260809 06:39:46.114225 11065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35251: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
I20260809 06:39:46.182533 11065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } attempt: 2
W20260809 06:39:46.184733 11065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35251: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
I20260809 06:39:46.276010 11065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } attempt: 3
W20260809 06:39:46.278189 11065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35251: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
W20260809 06:39:46.392556 11072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:46.392841 11072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:46.393023 11072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:46.400246 11072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:39:46.400367 11072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:46.400463 11072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:39:46.400563 11072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:39:46.400686 11072 flags.cc:425] Enabled experimental flag: --flush_threshold_secs=0
I20260809 06:39:46.409955 11072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=2000
--follower_unavailable_considered_failed_sec=4
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:38043,127.6.111.61:35251
--master_auto_join_cluster=false
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--flush_threshold_secs=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:39:46.410318 11072 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:38043,127.6.111.61:35251'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:39:46.411484 11072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:46.413244 11072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 06:39:46.421448 11065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } attempt: 4
W20260809 06:39:46.423563 11065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:35251: Network error: Client connection negotiation failed: client connection to 127.6.111.61:35251: connect: Connection refused (error 111)
W20260809 06:39:46.426016 11081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:46.426105 11085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:46.426064 11082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:46.427253 11072 server_base.cc:1034] running on GCE node
I20260809 06:39:46.428022 11072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:46.429477 11072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:46.430732 11072 hybrid_clock.cc:648] HybridClock initialized: now 1786257586430657 us; error 52 us; skew 500 ppm
I20260809 06:39:46.431057 11072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:46.433423 11072 webserver.cc:460] Webserver started at http://127.6.111.61:40467/ using document root <none> and password file <none>
I20260809 06:39:46.434146 11072 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:46.434298 11072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:46.434621 11072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:46.439895 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "28f4c5713cbd4d2d96252086ef806006"
format_stamp: "Formatted at 2026-08-09 06:39:46 on dist-test-slave-bw88"
I20260809 06:39:46.440711 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "28f4c5713cbd4d2d96252086ef806006"
format_stamp: "Formatted at 2026-08-09 06:39:46 on dist-test-slave-bw88"
I20260809 06:39:46.446131 11072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:39:46.449594 11091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:46.450963 11072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:39:46.451162 11072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "28f4c5713cbd4d2d96252086ef806006"
format_stamp: "Formatted at 2026-08-09 06:39:46 on dist-test-slave-bw88"
I20260809 06:39:46.451399 11072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:46.466920 11072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:46.467723 11072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:46.468039 11072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:46.486704 11072 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:35251
I20260809 06:39:46.486739 11159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:35251 every 8 connection(s)
I20260809 06:39:46.488037 11072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:39:46.488535 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 11072
I20260809 06:39:46.488742 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:39:46.492939 11161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:46.501806 11161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:46.512022 11161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:39:46.521759 11161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Bootstrap starting.
I20260809 06:39:46.525452 11161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:46.526508 11161 log.cc:824] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:46.529014 11161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: No bootstrap required, opened a new log
I20260809 06:39:46.536418 11161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.536763 11161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:46.536862 11161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f4c5713cbd4d2d96252086ef806006, State: Initialized, Role: FOLLOWER
I20260809 06:39:46.537568 11161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.538532 11168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.538882 11168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:46.539530 11161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:39:46.547906 11183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:46.548053 11183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:46.719026 11168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:46.719355 11168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
W20260809 06:39:46.720284 11098 tablet.cc:2369] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:46.720996 11168 leader_election.cc:288] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d3ee6fced0d4ee2ba409f58ea250645 (127.6.111.62:38043)
I20260809 06:39:46.722831 11065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } attempt: 5
I20260809 06:39:46.735657 11065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: Bootstrap starting.
I20260809 06:39:46.738889 11065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:46.740085 11065 log.cc:824] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:46.742862 11065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: No bootstrap required, opened a new log
I20260809 06:39:46.750576 11065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.750941 11065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:46.751080 11065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3ee6fced0d4ee2ba409f58ea250645, State: Initialized, Role: FOLLOWER
I20260809 06:39:46.751624 11065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.752609 11187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.752980 11187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:46.753809 11065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:39:46.753978 11034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28f4c5713cbd4d2d96252086ef806006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" is_pre_election: true
I20260809 06:39:46.754683 11034 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f4c5713cbd4d2d96252086ef806006 in term 0.
I20260809 06:39:46.755564 11096 leader_election.cc:302] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 28f4c5713cbd4d2d96252086ef806006, 5d3ee6fced0d4ee2ba409f58ea250645; no voters:
I20260809 06:39:46.756217 11168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:39:46.756372 11168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:39:46.756481 11168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:46.759902 11168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.760468 11168 leader_election.cc:288] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [CANDIDATE]: Term 1 election: Requested vote from peers 5d3ee6fced0d4ee2ba409f58ea250645 (127.6.111.62:38043)
I20260809 06:39:46.761056 11034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28f4c5713cbd4d2d96252086ef806006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3ee6fced0d4ee2ba409f58ea250645"
I20260809 06:39:46.761317 11034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 0 FOLLOWER]: Advancing to term 1
W20260809 06:39:46.761313 11004 tablet.cc:2369] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:46.764559 11034 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f4c5713cbd4d2d96252086ef806006 in term 1.
I20260809 06:39:46.765125 11096 leader_election.cc:302] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 28f4c5713cbd4d2d96252086ef806006, 5d3ee6fced0d4ee2ba409f58ea250645; no voters:
I20260809 06:39:46.765545 11168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 06:39:46.767937 11200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:39:46.768082 11200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:39:46.768481 11168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [term 1 LEADER]: Becoming Leader. State: Replica: 28f4c5713cbd4d2d96252086ef806006, State: Running, Role: LEADER
I20260809 06:39:46.769021 11168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } }
I20260809 06:39:46.772172 11202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f4c5713cbd4d2d96252086ef806006. Latest consensus state: current_term: 1 leader_uuid: "28f4c5713cbd4d2d96252086ef806006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.772574 11202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:46.777567 11206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:39:46.779321 11206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:39:46.789824 11034 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [term 1 FOLLOWER]: Refusing update from remote peer 28f4c5713cbd4d2d96252086ef806006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:39:46.790792 11202 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:39:46.800985 11187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f4c5713cbd4d2d96252086ef806006. Latest consensus state: current_term: 1 leader_uuid: "28f4c5713cbd4d2d96252086ef806006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.801287 11187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:46.805698 11202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28f4c5713cbd4d2d96252086ef806006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.805944 11202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006 [sys.catalog]: This master's current role is: LEADER
I20260809 06:39:46.807226 11206 catalog_manager.cc:1347] Generated new cluster ID: e3cd5c4a281343dbb77473773590719f
I20260809 06:39:46.807367 11206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:39:46.822122 11187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28f4c5713cbd4d2d96252086ef806006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d3ee6fced0d4ee2ba409f58ea250645" member_type: VOTER last_known_addr { host: "127.6.111.62" port: 38043 } } peers { permanent_uuid: "28f4c5713cbd4d2d96252086ef806006" member_type: VOTER last_known_addr { host: "127.6.111.61" port: 35251 } } }
I20260809 06:39:46.822363 11187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:39:46.837175 11206 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:39:46.839186 11206 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:39:46.853842 11206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Generated new TSK 0
I20260809 06:39:46.855295 11206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:39:46.861168 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.111.1:0
--local_ip_for_outbound_sockets=127.6.111.1
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:38043,127.6.111.61:35251
--builtin_ntp_servers=127.6.111.20:32993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 06:39:46.971781 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000120
I20260809 06:39:47.012681 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000072
I20260809 06:39:47.041812 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.069s user 0.026s sys 0.013s Metrics: {"bytes_written":6028,"cfile_init":1,"dirs.queue_time_us":6347,"dirs.run_cpu_time_us":185,"dirs.run_wall_time_us":1277,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1128,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":6480,"threads_started":1}
I20260809 06:39:47.043838 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260809 06:39:47.044755 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.087903 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.074s user 0.030s sys 0.018s Metrics: {"bytes_written":6028,"cfile_init":1,"dirs.queue_time_us":9988,"dirs.run_cpu_time_us":266,"dirs.run_wall_time_us":908,"drs_written":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1279,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":3,"thread_start_us":9802,"threads_started":1}
I20260809 06:39:47.089766 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260809 06:39:47.091859 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":239,"lbm_reads_lt_1ms":4}
W20260809 06:39:47.272615 11212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:39:47.272953 11212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:39:47.273109 11212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:39:47.280283 11212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:39:47.280493 11212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.111.1
I20260809 06:39:47.289423 11212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:32993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.111.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.111.1
--webserver_port=0
--tserver_master_addrs=127.6.111.62:38043,127.6.111.61:35251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.111.1
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
I20260809 06:39:47.290841 11212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:39:47.292743 11212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:39:47.306268 11222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:47.306247 11226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:39:47.306352 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:39:47.306995 11212 server_base.cc:1034] running on GCE node
I20260809 06:39:47.307646 11212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:39:47.308565 11212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:39:47.309820 11212 hybrid_clock.cc:648] HybridClock initialized: now 1786257587309753 us; error 43 us; skew 500 ppm
I20260809 06:39:47.310129 11212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:39:47.312309 11212 webserver.cc:460] Webserver started at http://127.6.111.1:44519/ using document root <none> and password file <none>
I20260809 06:39:47.313009 11212 fs_manager.cc:359] Metadata directory not provided
I20260809 06:39:47.313158 11212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:39:47.313545 11212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:39:47.316243 11212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data/instance:
uuid: "8f4f2c9d8f334254a624a536624a1027"
format_stamp: "Formatted at 2026-08-09 06:39:47 on dist-test-slave-bw88"
I20260809 06:39:47.317025 11212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance:
uuid: "8f4f2c9d8f334254a624a536624a1027"
format_stamp: "Formatted at 2026-08-09 06:39:47 on dist-test-slave-bw88"
I20260809 06:39:47.322347 11212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:39:47.325855 11233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:47.327122 11212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:39:47.327302 11212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
uuid: "8f4f2c9d8f334254a624a536624a1027"
format_stamp: "Formatted at 2026-08-09 06:39:47 on dist-test-slave-bw88"
I20260809 06:39:47.327507 11212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:39:47.346915 11212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:39:47.347724 11212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:39:47.348024 11212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:39:47.349005 11212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:39:47.350780 11212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:39:47.350888 11212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:47.351043 11212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:39:47.351116 11212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:39:47.385149 11212 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.1:43773
I20260809 06:39:47.385198 11378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.1:43773 every 8 connection(s)
I20260809 06:39:47.386538 11212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/data/info.pb
I20260809 06:39:47.396108 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 11212
I20260809 06:39:47.396327 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/instance
I20260809 06:39:47.404487 11379 heartbeater.cc:347] Connected to a master server at 127.6.111.61:35251
I20260809 06:39:47.404619 11381 heartbeater.cc:347] Connected to a master server at 127.6.111.62:38043
I20260809 06:39:47.404850 11381 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:47.404853 11379 heartbeater.cc:464] Registering TS with master...
I20260809 06:39:47.405776 11381 heartbeater.cc:511] Master 127.6.111.62:38043 requested a full tablet report, sending...
I20260809 06:39:47.405797 11379 heartbeater.cc:511] Master 127.6.111.61:35251 requested a full tablet report, sending...
I20260809 06:39:47.407773 11114 ts_manager.cc:194] Registered new tserver with Master: 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1:43773)
I20260809 06:39:47.407965 11021 ts_manager.cc:194] Registered new tserver with Master: 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1:43773)
I20260809 06:39:47.409495 11114 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.111.1:49241
I20260809 06:39:47.416361 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:47.421278 6588 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20260809 06:39:47.461079 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53234:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.492126 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 4df87f0296984d5491a184de2a3e8b15 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.1 [id=d79cd38841184b33ab875b80901ed8eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.493821 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df87f0296984d5491a184de2a3e8b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.504848 11411 tablet_bootstrap.cc:492] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.507740 11411 tablet_bootstrap.cc:654] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.508713 11411 log.cc:824] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: Log is configured to *not* fsync() on all Append() calls
I20260809 06:39:47.511416 11411 tablet_bootstrap.cc:492] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.511628 11411 ts_tablet_manager.cc:1397] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 06:39:47.518272 11411 raft_consensus.cc:348] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.518569 11411 raft_consensus.cc:374] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.518646 11411 raft_consensus.cc:729] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.519238 11411 consensus_queue.cc:260] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.519529 11411 raft_consensus.cc:388] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.519680 11411 raft_consensus.cc:482] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.519876 11411 raft_consensus.cc:3037] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.521854 11411 raft_consensus.cc:504] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.522303 11411 leader_election.cc:302] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.522792 11411 leader_election.cc:288] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.522950 11413 raft_consensus.cc:2781] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.523705 11413 raft_consensus.cc:686] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.523910 11379 heartbeater.cc:503] Master 127.6.111.61:35251 was elected leader, sending a full tablet report...
I20260809 06:39:47.524290 11413 consensus_queue.cc:237] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.524658 11411 ts_tablet_manager.cc:1428] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260809 06:39:47.529748 11114 catalog_manager.cc:5495] T 4df87f0296984d5491a184de2a3e8b15 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.546137 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000140
I20260809 06:39:47.558346 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.006s Metrics: {"bytes_written":5192,"cfile_init":1,"dirs.queue_time_us":394,"dirs.run_cpu_time_us":276,"dirs.run_wall_time_us":1720,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":727,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"thread_start_us":246,"threads_started":1}
I20260809 06:39:47.560097 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260809 06:39:47.567497 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53248:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.579931 11297 tablet_service.cc:1467] Processing CreateTablet for tablet af1ef4bbbac5420fb71e380c32ec9be3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.2 [id=06d734a34d3d4963aabf0e721582f2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.580466 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1ef4bbbac5420fb71e380c32ec9be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.585186 11411 tablet_bootstrap.cc:492] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.586951 11411 tablet_bootstrap.cc:654] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.589716 11411 tablet_bootstrap.cc:492] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.589900 11411 ts_tablet_manager.cc:1397] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:47.591181 11411 raft_consensus.cc:348] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.591431 11411 raft_consensus.cc:374] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.591548 11411 raft_consensus.cc:729] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.591838 11411 consensus_queue.cc:260] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.592147 11411 raft_consensus.cc:388] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.592244 11411 raft_consensus.cc:482] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.592375 11411 raft_consensus.cc:3037] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.594018 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000141
I20260809 06:39:47.596006 11411 raft_consensus.cc:504] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.596385 11411 leader_election.cc:302] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.596626 11411 leader_election.cc:288] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.596735 11413 raft_consensus.cc:2781] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.596920 11413 raft_consensus.cc:686] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.597169 11413 consensus_queue.cc:237] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.597540 11411 ts_tablet_manager.cc:1428] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 06:39:47.598165 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.021s sys 0.016s Metrics: {"bytes_written":6488,"cfile_cache_hit":12,"cfile_cache_hit_bytes":1914,"cfile_cache_miss":28,"cfile_cache_miss_bytes":2950,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":2142,"dirs.run_cpu_time_us":1194,"dirs.run_wall_time_us":6129,"drs_written":1,"lbm_read_time_us":1037,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1086,"lbm_writes_lt_1ms":27,"mutex_wait_us":19,"num_input_rowsets":2,"peak_mem_usage":50,"rows_written":5,"spinlock_wait_cycles":4864,"thread_start_us":10983,"threads_started":3}
I20260809 06:39:47.599124 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:47.600194 11114 catalog_manager.cc:5495] T af1ef4bbbac5420fb71e380c32ec9be3 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.613549 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.013s sys 0.000s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":1302,"dirs.run_cpu_time_us":148,"dirs.run_wall_time_us":1529,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1101,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:47.614594 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.461111
I20260809 06:39:47.620441 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.011s sys 0.008s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":8676,"dirs.run_cpu_time_us":233,"dirs.run_wall_time_us":1052,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1255,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":4,"thread_start_us":8778,"threads_started":1,"update_count":1}
I20260809 06:39:47.621680 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.308333
I20260809 06:39:47.626282 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.011s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":573,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":115,"lbm_read_time_us":451,"lbm_reads_lt_1ms":25,"lbm_write_time_us":300,"lbm_writes_lt_1ms":9,"mutex_wait_us":21,"reinsert_count":0,"spinlock_wait_cycles":1280,"update_count":1}
I20260809 06:39:47.626901 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 626 bytes on disk
I20260809 06:39:47.627640 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":134,"lbm_reads_lt_1ms":8}
I20260809 06:39:47.628212 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.360132
I20260809 06:39:47.643688 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.020s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":869,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":966,"dirs.run_cpu_time_us":281,"dirs.run_wall_time_us":1546,"lbm_read_time_us":559,"lbm_reads_lt_1ms":25,"lbm_write_time_us":372,"lbm_writes_lt_1ms":9,"reinsert_count":0,"thread_start_us":5728,"threads_started":1,"update_count":1}
I20260809 06:39:47.644233 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260809 06:39:47.650599 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.013s sys 0.008s Metrics: {"bytes_written":6917,"cfile_cache_hit":13,"cfile_cache_hit_bytes":2317,"cfile_cache_miss":31,"cfile_cache_miss_bytes":3895,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1119,"dirs.run_cpu_time_us":1061,"dirs.run_wall_time_us":5304,"drs_written":1,"lbm_read_time_us":1251,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1009,"lbm_writes_lt_1ms":27,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":303,"rows_written":7,"thread_start_us":552,"threads_started":1}
I20260809 06:39:47.651480 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 472 bytes on disk
I20260809 06:39:47.651795 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53254:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.652209 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.665158 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5ea9e7df374835b7cd1205ee9b6353 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.3 [id=e7c98bc6d00c4b48a25f03d20e135db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.665697 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5ea9e7df374835b7cd1205ee9b6353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.669901 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.019s sys 0.006s Metrics: {"bytes_written":6917,"cfile_cache_hit":13,"cfile_cache_hit_bytes":2307,"cfile_cache_miss":31,"cfile_cache_miss_bytes":3863,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1653,"dirs.run_cpu_time_us":945,"dirs.run_wall_time_us":4975,"drs_written":1,"lbm_read_time_us":1602,"lbm_reads_lt_1ms":67,"lbm_write_time_us":812,"lbm_writes_lt_1ms":27,"mutex_wait_us":50,"num_input_rowsets":2,"peak_mem_usage":50,"rows_written":7,"thread_start_us":4106,"threads_started":3}
I20260809 06:39:47.670792 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 472 bytes on disk
I20260809 06:39:47.671332 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.671878 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000014
I20260809 06:39:47.672487 11411 tablet_bootstrap.cc:492] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.673902 11411 tablet_bootstrap.cc:654] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.675906 11411 tablet_bootstrap.cc:492] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.676052 11411 ts_tablet_manager.cc:1397] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:47.676748 11411 raft_consensus.cc:348] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.676954 11411 raft_consensus.cc:374] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.677064 11411 raft_consensus.cc:729] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.677379 11411 consensus_queue.cc:260] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.677541 11411 raft_consensus.cc:388] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.677659 11411 raft_consensus.cc:482] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.677765 11411 raft_consensus.cc:3037] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.680604 11411 raft_consensus.cc:504] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.680857 11411 leader_election.cc:302] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.681123 11411 leader_election.cc:288] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.681187 11413 raft_consensus.cc:2781] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.681639 11411 ts_tablet_manager.cc:1428] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260809 06:39:47.681821 11413 raft_consensus.cc:686] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.682097 11413 consensus_queue.cc:237] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.684607 11114 catalog_manager.cc:5495] T 1d5ea9e7df374835b7cd1205ee9b6353 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.685098 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.005s Metrics: {"bytes_written":5155,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":1344,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":797,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:47.686031 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.345041
I20260809 06:39:47.705819 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.012s sys 0.007s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":2605,"cfile_cache_miss":28,"cfile_cache_miss_bytes":3959,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":849,"dirs.run_cpu_time_us":634,"dirs.run_wall_time_us":3522,"drs_written":1,"lbm_read_time_us":925,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1003,"lbm_writes_lt_1ms":31,"mutex_wait_us":84,"num_input_rowsets":2,"peak_mem_usage":557,"reinsert_count":0,"rows_written":9,"spinlock_wait_cycles":2816,"thread_start_us":399,"threads_started":1,"update_count":1}
I20260809 06:39:47.706841 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 540 bytes on disk
I20260809 06:39:47.707540 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.708182 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.120871
I20260809 06:39:47.716593 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2422,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":145,"lbm_read_time_us":442,"lbm_reads_lt_1ms":21,"lbm_write_time_us":338,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260809 06:39:47.716885 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53268:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.727355 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 99540fc0c6ac4790aa35b6ba257fcfe7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.4 [id=66eeb9c9db884543b550d3aea88b53a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.727854 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99540fc0c6ac4790aa35b6ba257fcfe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.732654 11411 tablet_bootstrap.cc:492] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.734031 11411 tablet_bootstrap.cc:654] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.735984 11411 tablet_bootstrap.cc:492] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.736138 11411 ts_tablet_manager.cc:1397] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:47.736778 11411 raft_consensus.cc:348] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.736948 11411 raft_consensus.cc:374] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.737061 11411 raft_consensus.cc:729] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.737296 11411 consensus_queue.cc:260] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.737576 11411 raft_consensus.cc:388] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.737679 11411 raft_consensus.cc:482] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.737787 11411 raft_consensus.cc:3037] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.739831 11411 raft_consensus.cc:504] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.740077 11411 leader_election.cc:302] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.740257 11411 leader_election.cc:288] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.740367 11413 raft_consensus.cc:2781] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.740582 11413 raft_consensus.cc:686] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.740767 11411 ts_tablet_manager.cc:1428] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:47.740844 11413 consensus_queue.cc:237] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.743709 11113 catalog_manager.cc:5495] T 99540fc0c6ac4790aa35b6ba257fcfe7 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.771420 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53274:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.772629 11200 catalog_manager.cc:1259] Loaded cluster ID: e3cd5c4a281343dbb77473773590719f
I20260809 06:39:47.772754 11200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: loading cluster ID for follower catalog manager: success
I20260809 06:39:47.777446 11200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: acquiring CA information for follower catalog manager: success
I20260809 06:39:47.780966 11200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d3ee6fced0d4ee2ba409f58ea250645: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:39:47.782552 11297 tablet_service.cc:1467] Processing CreateTablet for tablet f39ffe23f2c04dfc9f95da9f0989bf41 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.5 [id=024b7b7a94f6490eaa13032ae7df2a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.782986 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f39ffe23f2c04dfc9f95da9f0989bf41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.787822 11411 tablet_bootstrap.cc:492] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.789081 11411 tablet_bootstrap.cc:654] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.791712 11411 tablet_bootstrap.cc:492] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.791877 11411 ts_tablet_manager.cc:1397] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:47.792716 11411 raft_consensus.cc:348] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.792940 11411 raft_consensus.cc:374] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.793035 11411 raft_consensus.cc:729] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.793303 11411 consensus_queue.cc:260] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.793530 11411 raft_consensus.cc:388] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.793642 11411 raft_consensus.cc:482] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.793748 11411 raft_consensus.cc:3037] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.796605 11411 raft_consensus.cc:504] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.796877 11411 leader_election.cc:302] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.797137 11411 leader_election.cc:288] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.797243 11413 raft_consensus.cc:2781] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.797444 11413 raft_consensus.cc:686] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.797658 11411 ts_tablet_manager.cc:1428] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 06:39:47.797693 11413 consensus_queue.cc:237] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.800101 11113 catalog_manager.cc:5495] T f39ffe23f2c04dfc9f95da9f0989bf41 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.840677 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53282:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.852626 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a30e09f43b544d2c8c95e290a6b9ee8b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.6 [id=de340c8381b64f11a5cafce8eeb6d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.853108 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a30e09f43b544d2c8c95e290a6b9ee8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.859851 11411 tablet_bootstrap.cc:492] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.861596 11411 tablet_bootstrap.cc:654] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.864071 11411 tablet_bootstrap.cc:492] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.864228 11411 ts_tablet_manager.cc:1397] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 06:39:47.865023 11411 raft_consensus.cc:348] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.865200 11411 raft_consensus.cc:374] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.865324 11411 raft_consensus.cc:729] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.865594 11411 consensus_queue.cc:260] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.865866 11411 raft_consensus.cc:388] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.865984 11411 raft_consensus.cc:482] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.866098 11411 raft_consensus.cc:3037] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.869318 11411 raft_consensus.cc:504] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.869614 11411 leader_election.cc:302] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.869832 11411 leader_election.cc:288] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.869935 11413 raft_consensus.cc:2781] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.870103 11413 raft_consensus.cc:686] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.870337 11411 ts_tablet_manager.cc:1428] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 06:39:47.870339 11413 consensus_queue.cc:237] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.875460 11113 catalog_manager.cc:5495] T a30e09f43b544d2c8c95e290a6b9ee8b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.903249 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:47.909436 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53292:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:47.915369 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.010s sys 0.000s Metrics: {"bytes_written":6488,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1389,"drs_written":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4,"lbm_write_time_us":797,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:47.916272 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 601 bytes on disk
I20260809 06:39:47.916888 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.917761 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.560648
I20260809 06:39:47.922101 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 98e483b0d3e74b2584514643b2d3a426 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.7 [id=31b9b6d605d0427b84c9fff7ab861400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:47.922575 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98e483b0d3e74b2584514643b2d3a426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:47.927255 11411 tablet_bootstrap.cc:492] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:47.928758 11411 tablet_bootstrap.cc:654] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:47.930739 11411 tablet_bootstrap.cc:492] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:47.930914 11411 ts_tablet_manager.cc:1397] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:47.931586 11411 raft_consensus.cc:348] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.931799 11411 raft_consensus.cc:374] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:47.931883 11411 raft_consensus.cc:729] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:47.932117 11411 consensus_queue.cc:260] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.932298 11411 raft_consensus.cc:388] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:47.932402 11411 raft_consensus.cc:482] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:47.932515 11411 raft_consensus.cc:3037] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:47.934725 11411 raft_consensus.cc:504] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.934991 11411 leader_election.cc:302] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:47.934916 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.016s sys 0.000s Metrics: {"bytes_written":8680,"cfile_cache_hit":12,"cfile_cache_hit_bytes":3732,"cfile_cache_miss":28,"cfile_cache_miss_bytes":5814,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1070,"dirs.run_cpu_time_us":707,"dirs.run_wall_time_us":3827,"drs_written":1,"lbm_read_time_us":817,"lbm_reads_lt_1ms":56,"lbm_write_time_us":791,"lbm_writes_lt_1ms":27,"mutex_wait_us":179,"num_input_rowsets":2,"peak_mem_usage":557,"rows_written":15,"thread_start_us":729,"threads_started":2}
I20260809 06:39:47.935681 11413 raft_consensus.cc:2781] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:47.935844 11413 raft_consensus.cc:686] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:47.935973 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000006
I20260809 06:39:47.936095 11413 consensus_queue.cc:237] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:47.936873 11411 leader_election.cc:288] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:47.937471 11411 ts_tablet_manager.cc:1428] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 06:39:47.940343 11112 catalog_manager.cc:5495] T 98e483b0d3e74b2584514643b2d3a426 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:47.947722 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"bytes_written":5155,"cfile_init":1,"dirs.queue_time_us":2107,"dirs.run_cpu_time_us":261,"dirs.run_wall_time_us":1373,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":991,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:47.948472 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1031 bytes on disk
I20260809 06:39:47.949106 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":108,"lbm_reads_lt_1ms":8}
I20260809 06:39:47.949743 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.310508
I20260809 06:39:47.967418 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:47.968459 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.014s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":4072,"cfile_cache_miss":28,"cfile_cache_miss_bytes":6128,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":648,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1054,"drs_written":1,"lbm_read_time_us":913,"lbm_reads_lt_1ms":56,"lbm_write_time_us":793,"lbm_writes_lt_1ms":31,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":1569,"reinsert_count":0,"rows_written":17,"spinlock_wait_cycles":7936,"update_count":1}
I20260809 06:39:47.969581 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 880 bytes on disk
I20260809 06:39:47.970211 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260809 06:39:47.980608 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":6487,"cfile_init":1,"dirs.queue_time_us":203,"dirs.run_cpu_time_us":146,"dirs.run_wall_time_us":1039,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":957,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:47.981513 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.530665
I20260809 06:39:47.988329 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53304:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.001890 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf84d6fa20d47fdb49714c4e2a3df73 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.8 [id=96b97e68b7fe4bd6a06a88812c461a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.002391 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf84d6fa20d47fdb49714c4e2a3df73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.005530 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.018s sys 0.001s Metrics: {"bytes_written":9140,"cfile_cache_hit":24,"cfile_cache_hit_bytes":6904,"cfile_cache_miss":20,"cfile_cache_miss_bytes":4082,"cfile_init":5,"delta_iterators_relevant":6,"dirs.queue_time_us":854,"dirs.run_cpu_time_us":1023,"dirs.run_wall_time_us":7526,"drs_written":1,"lbm_read_time_us":777,"lbm_reads_lt_1ms":40,"lbm_write_time_us":799,"lbm_writes_lt_1ms":27,"mutex_wait_us":99,"num_input_rowsets":2,"peak_mem_usage":825,"rows_written":17,"thread_start_us":4965,"threads_started":1}
I20260809 06:39:48.006353 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 908 bytes on disk
I20260809 06:39:48.006875 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.007046 11411 tablet_bootstrap.cc:492] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.007426 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260809 06:39:48.008345 11411 tablet_bootstrap.cc:654] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.010565 11411 tablet_bootstrap.cc:492] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.010727 11411 ts_tablet_manager.cc:1397] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:48.011364 11411 raft_consensus.cc:348] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.011572 11411 raft_consensus.cc:374] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.011690 11411 raft_consensus.cc:729] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.011931 11411 consensus_queue.cc:260] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.012130 11411 raft_consensus.cc:388] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.012214 11411 raft_consensus.cc:482] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.012327 11411 raft_consensus.cc:3037] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.014540 11411 raft_consensus.cc:504] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.014806 11411 leader_election.cc:302] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.015036 11411 leader_election.cc:288] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.015100 11413 raft_consensus.cc:2781] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.015283 11413 raft_consensus.cc:686] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.015589 11411 ts_tablet_manager.cc:1428] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260809 06:39:48.015520 11413 consensus_queue.cc:237] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.017896 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.001s Metrics: {"bytes_written":5155,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":190,"dirs.run_wall_time_us":3630,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":771,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:48.018368 11113 catalog_manager.cc:5495] T 9bf84d6fa20d47fdb49714c4e2a3df73 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.018761 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:48.019392 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.020074 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.297199
I20260809 06:39:48.044284 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.022s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":4444,"cfile_cache_miss":28,"cfile_cache_miss_bytes":6758,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":452,"dirs.run_cpu_time_us":766,"dirs.run_wall_time_us":4619,"drs_written":1,"lbm_read_time_us":1115,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1136,"lbm_writes_lt_1ms":31,"mutex_wait_us":283,"num_input_rowsets":2,"peak_mem_usage":2026,"reinsert_count":0,"rows_written":19,"update_count":1}
I20260809 06:39:48.045317 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 976 bytes on disk
I20260809 06:39:48.045977 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.058748 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53306:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.069900 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa56670be3541c7ab322f77448c3ae1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.9 [id=469d33cb2a21467ebe107e37a1a373ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.070356 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa56670be3541c7ab322f77448c3ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.074936 11411 tablet_bootstrap.cc:492] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.076260 11411 tablet_bootstrap.cc:654] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.078397 11411 tablet_bootstrap.cc:492] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.078575 11411 ts_tablet_manager.cc:1397] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:48.079178 11411 raft_consensus.cc:348] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.079370 11411 raft_consensus.cc:374] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.079476 11411 raft_consensus.cc:729] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.079743 11411 consensus_queue.cc:260] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.079913 11411 raft_consensus.cc:388] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.080039 11411 raft_consensus.cc:482] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.080158 11411 raft_consensus.cc:3037] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.082124 11411 raft_consensus.cc:504] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.082358 11411 leader_election.cc:302] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.082572 11411 leader_election.cc:288] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.082679 11413 raft_consensus.cc:2781] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.082867 11413 raft_consensus.cc:686] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.083035 11411 ts_tablet_manager.cc:1428] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.083149 11413 consensus_queue.cc:237] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.085762 11113 catalog_manager.cc:5495] T 3fa56670be3541c7ab322f77448c3ae1 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.117128 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53312:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.128216 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 19aee9b722e6427f821d7498aeb8e10b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.10 [id=74e3d5009a7c4e338db8c2b22a0ab562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.128825 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19aee9b722e6427f821d7498aeb8e10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.133948 11411 tablet_bootstrap.cc:492] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.135373 11411 tablet_bootstrap.cc:654] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.137259 11411 tablet_bootstrap.cc:492] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.137427 11411 ts_tablet_manager.cc:1397] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.138032 11411 raft_consensus.cc:348] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.138213 11411 raft_consensus.cc:374] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.138283 11411 raft_consensus.cc:729] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.138511 11411 consensus_queue.cc:260] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.138677 11411 raft_consensus.cc:388] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.138736 11411 raft_consensus.cc:482] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.138813 11411 raft_consensus.cc:3037] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.140936 11411 raft_consensus.cc:504] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.141247 11411 leader_election.cc:302] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.141497 11411 leader_election.cc:288] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.141587 11413 raft_consensus.cc:2781] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.141776 11413 raft_consensus.cc:686] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.141961 11411 ts_tablet_manager.cc:1428] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260809 06:39:48.141992 11413 consensus_queue.cc:237] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.144614 11113 catalog_manager.cc:5495] T 19aee9b722e6427f821d7498aeb8e10b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.173676 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53316:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.184511 11297 tablet_service.cc:1467] Processing CreateTablet for tablet cd5526e40dd1417cb73f905bb0e6eafd (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.11 [id=83930c87d7854cc5b9fc072bf887f289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.184916 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5526e40dd1417cb73f905bb0e6eafd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.189759 11411 tablet_bootstrap.cc:492] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.191067 11411 tablet_bootstrap.cc:654] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.193132 11411 tablet_bootstrap.cc:492] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.193306 11411 ts_tablet_manager.cc:1397] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:48.193938 11411 raft_consensus.cc:348] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.194137 11411 raft_consensus.cc:374] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.194237 11411 raft_consensus.cc:729] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.194484 11411 consensus_queue.cc:260] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.194664 11411 raft_consensus.cc:388] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.194793 11411 raft_consensus.cc:482] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.194905 11411 raft_consensus.cc:3037] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.196967 11411 raft_consensus.cc:504] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.197224 11411 leader_election.cc:302] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.197455 11411 leader_election.cc:288] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.197508 11413 raft_consensus.cc:2781] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.197764 11413 raft_consensus.cc:686] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.198006 11411 ts_tablet_manager.cc:1428] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:48.198029 11413 consensus_queue.cc:237] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.200508 11113 catalog_manager.cc:5495] T cd5526e40dd1417cb73f905bb0e6eafd P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.221042 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000076
I20260809 06:39:48.229516 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53322:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.234297 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.003s Metrics: {"bytes_written":6490,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":1394,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1042,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:48.235208 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 599 bytes on disk
I20260809 06:39:48.235823 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":105,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.236514 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.437666
I20260809 06:39:48.242587 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 40404dc9acab4f448836202dd7fc6d2b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.12 [id=871df8da8fdd45e683e0816513ef97ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.243114 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40404dc9acab4f448836202dd7fc6d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.248423 11411 tablet_bootstrap.cc:492] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.250254 11411 tablet_bootstrap.cc:654] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.252524 11411 tablet_bootstrap.cc:492] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.252717 11411 ts_tablet_manager.cc:1397] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.253428 11411 raft_consensus.cc:348] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.253664 11411 raft_consensus.cc:374] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.253777 11411 raft_consensus.cc:729] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.254060 11411 consensus_queue.cc:260] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.254285 11411 raft_consensus.cc:388] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.254379 11411 raft_consensus.cc:482] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.254493 11411 raft_consensus.cc:3037] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.256538 11411 raft_consensus.cc:504] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.256803 11411 leader_election.cc:302] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.257018 11411 leader_election.cc:288] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.257107 11413 raft_consensus.cc:2781] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.257302 11413 raft_consensus.cc:686] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.257519 11411 ts_tablet_manager.cc:1428] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:48.257596 11413 consensus_queue.cc:237] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.257972 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.017s sys 0.003s Metrics: {"bytes_written":10901,"cfile_cache_hit":13,"cfile_cache_hit_bytes":5604,"cfile_cache_miss":31,"cfile_cache_miss_bytes":8982,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1105,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1446,"drs_written":1,"lbm_read_time_us":1437,"lbm_reads_lt_1ms":63,"lbm_write_time_us":844,"lbm_writes_lt_1ms":27,"mutex_wait_us":20,"num_input_rowsets":2,"peak_mem_usage":2045,"rows_written":25,"spinlock_wait_cycles":1280}
I20260809 06:39:48.258947 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1237 bytes on disk
I20260809 06:39:48.260751 11112 catalog_manager.cc:5495] T 40404dc9acab4f448836202dd7fc6d2b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.261168 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.261893 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260809 06:39:48.274718 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.008s sys 0.003s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":79,"dirs.run_cpu_time_us":197,"dirs.run_wall_time_us":1272,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1047,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:48.275704 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:48.276368 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.277027 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.460472
I20260809 06:39:48.285471 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":574,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":118,"lbm_read_time_us":416,"lbm_reads_lt_1ms":21,"lbm_write_time_us":277,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260809 06:39:48.286159 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.303633
I20260809 06:39:48.296001 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53330:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.296862 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260809 06:39:48.310614 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d917e7b5aa7a4eebb386211eb3d7e1e1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.13 [id=5d2ff2600bbf497d8e815c1144c7e044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.311149 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d917e7b5aa7a4eebb386211eb3d7e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.312223 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.010s sys 0.000s Metrics: {"bytes_written":6490,"cfile_init":1,"dirs.queue_time_us":86,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":1428,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":964,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:48.312985 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.426554
I20260809 06:39:48.314030 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.018s sys 0.003s Metrics: {"bytes_written":11330,"cfile_cache_hit":13,"cfile_cache_hit_bytes":5986,"cfile_cache_miss":31,"cfile_cache_miss_bytes":9382,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":489,"dirs.run_cpu_time_us":731,"dirs.run_wall_time_us":9489,"drs_written":1,"lbm_read_time_us":1332,"lbm_reads_lt_1ms":63,"lbm_write_time_us":825,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":3038,"rows_written":27}
I20260809 06:39:48.314881 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1315 bytes on disk
I20260809 06:39:48.315598 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.316191 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260809 06:39:48.317518 11411 tablet_bootstrap.cc:492] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.319037 11411 tablet_bootstrap.cc:654] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.321027 11411 tablet_bootstrap.cc:492] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.321238 11411 ts_tablet_manager.cc:1397] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.322314 11411 raft_consensus.cc:348] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.322564 11411 raft_consensus.cc:374] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.322753 11411 raft_consensus.cc:729] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.323036 11411 consensus_queue.cc:260] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.323258 11411 raft_consensus.cc:388] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.323370 11411 raft_consensus.cc:482] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.323498 11411 raft_consensus.cc:3037] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.325654 11411 raft_consensus.cc:504] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.325893 11411 leader_election.cc:302] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.326231 11413 raft_consensus.cc:2781] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.326390 11413 raft_consensus.cc:686] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.326655 11413 consensus_queue.cc:237] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.328440 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.004s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":58,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1217,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":922,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:48.329200 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.290949
I20260809 06:39:48.329455 11411 leader_election.cc:288] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.330262 11411 ts_tablet_manager.cc:1428] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260809 06:39:48.334950 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.021s sys 0.000s Metrics: {"bytes_written":11330,"cfile_cache_hit":13,"cfile_cache_hit_bytes":5933,"cfile_cache_miss":31,"cfile_cache_miss_bytes":9499,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":526,"dirs.run_cpu_time_us":202,"dirs.run_wall_time_us":1199,"drs_written":1,"lbm_read_time_us":1319,"lbm_reads_lt_1ms":67,"lbm_write_time_us":963,"lbm_writes_lt_1ms":27,"mutex_wait_us":60,"num_input_rowsets":2,"peak_mem_usage":2332,"rows_written":27,"spinlock_wait_cycles":11008}
I20260809 06:39:48.335750 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1315 bytes on disk
I20260809 06:39:48.336270 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.336735 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260809 06:39:48.340760 11113 catalog_manager.cc:5495] T d917e7b5aa7a4eebb386211eb3d7e1e1 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.348425 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":160,"dirs.run_wall_time_us":3237,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1054,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:48.349403 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:48.349977 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.350653 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.290949
I20260809 06:39:48.354354 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.014s sys 0.009s Metrics: {"bytes_written":11733,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6278,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9444,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":887,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":2097,"drs_written":1,"lbm_read_time_us":1397,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1246,"lbm_writes_lt_1ms":27,"mutex_wait_us":50,"num_input_rowsets":2,"peak_mem_usage":3291,"rows_written":29,"spinlock_wait_cycles":3840}
I20260809 06:39:48.355475 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260809 06:39:48.361382 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.000s Metrics: {"bytes_written":330,"delete_count":0,"lbm_write_time_us":129,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:48.361985 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260809 06:39:48.366739 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":156,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:48.367187 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1385 bytes on disk
I20260809 06:39:48.367650 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.368150 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.102536
I20260809 06:39:48.371959 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.019s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":6278,"cfile_cache_miss":28,"cfile_cache_miss_bytes":9444,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":341,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1842,"drs_written":1,"lbm_read_time_us":1282,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1071,"lbm_writes_lt_1ms":31,"mutex_wait_us":64,"num_input_rowsets":2,"peak_mem_usage":3291,"reinsert_count":0,"rows_written":29,"update_count":1}
I20260809 06:39:48.372614 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1396 bytes on disk
I20260809 06:39:48.373152 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.378106 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5656,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":181,"lbm_read_time_us":563,"lbm_reads_lt_1ms":27,"lbm_write_time_us":347,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":2}
I20260809 06:39:48.378683 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:48.378917 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 06:39:48.390319 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53334:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.402499 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 8df7713c8cbd431d8082c1040e666293 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.14 [id=3c3894f5ed8e4b86883d4076137d2e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.402935 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df7713c8cbd431d8082c1040e666293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.408252 11411 tablet_bootstrap.cc:492] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.409677 11411 tablet_bootstrap.cc:654] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.415763 11411 tablet_bootstrap.cc:492] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.415912 11411 ts_tablet_manager.cc:1397] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 06:39:48.416642 11411 raft_consensus.cc:348] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.416800 11411 raft_consensus.cc:374] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.416918 11411 raft_consensus.cc:729] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.417160 11411 consensus_queue.cc:260] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.417392 11411 raft_consensus.cc:388] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.417480 11411 raft_consensus.cc:482] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.417632 11411 raft_consensus.cc:3037] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.419808 11411 raft_consensus.cc:504] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.420112 11411 leader_election.cc:302] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.420400 11411 leader_election.cc:288] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.420487 11413 raft_consensus.cc:2781] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.420635 11413 raft_consensus.cc:686] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.420904 11411 ts_tablet_manager.cc:1428] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:48.420852 11413 consensus_queue.cc:237] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.424230 11113 catalog_manager.cc:5495] T 8df7713c8cbd431d8082c1040e666293 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.460136 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53346:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.470642 11297 tablet_service.cc:1467] Processing CreateTablet for tablet da1b79beffff487490eef4796568f69a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.15 [id=18df979f58c3493793bcbed178938f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.471125 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1b79beffff487490eef4796568f69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.475735 11411 tablet_bootstrap.cc:492] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.477030 11411 tablet_bootstrap.cc:654] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.479302 11411 tablet_bootstrap.cc:492] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.479487 11411 ts_tablet_manager.cc:1397] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.480072 11411 raft_consensus.cc:348] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.480275 11411 raft_consensus.cc:374] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.480373 11411 raft_consensus.cc:729] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.480635 11411 consensus_queue.cc:260] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.480805 11411 raft_consensus.cc:388] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.480901 11411 raft_consensus.cc:482] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.481027 11411 raft_consensus.cc:3037] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.482874 11411 raft_consensus.cc:504] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.483119 11411 leader_election.cc:302] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.483361 11411 leader_election.cc:288] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.483421 11413 raft_consensus.cc:2781] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.483657 11413 raft_consensus.cc:686] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.483852 11411 ts_tablet_manager.cc:1428] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.483923 11413 consensus_queue.cc:237] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.486860 11113 catalog_manager.cc:5495] T da1b79beffff487490eef4796568f69a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.515342 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53362:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.526156 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 7994c1d60ee04a1fa7798346d731a16c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.16 [id=c617456f382941dd9ef0e6c390985f86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.526615 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7994c1d60ee04a1fa7798346d731a16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.531401 11411 tablet_bootstrap.cc:492] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.532720 11411 tablet_bootstrap.cc:654] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.534675 11411 tablet_bootstrap.cc:492] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.534788 11411 ts_tablet_manager.cc:1397] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 06:39:48.535372 11411 raft_consensus.cc:348] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.535528 11411 raft_consensus.cc:374] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.535596 11411 raft_consensus.cc:729] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.535809 11411 consensus_queue.cc:260] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.535964 11411 raft_consensus.cc:388] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.536027 11411 raft_consensus.cc:482] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.536105 11411 raft_consensus.cc:3037] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.537876 11411 raft_consensus.cc:504] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.538081 11411 leader_election.cc:302] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.538216 11411 leader_election.cc:288] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.538322 11413 raft_consensus.cc:2781] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.538566 11413 raft_consensus.cc:686] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.538782 11411 ts_tablet_manager.cc:1428] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.538789 11413 consensus_queue.cc:237] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.541754 11112 catalog_manager.cc:5495] T 7994c1d60ee04a1fa7798346d731a16c P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.570559 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53376:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.581801 11297 tablet_service.cc:1467] Processing CreateTablet for tablet e3ce67ed6a5d4d76a97d07e7ddc951fc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.17 [id=abe80978e25545ff94c165af44e14456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.582324 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ce67ed6a5d4d76a97d07e7ddc951fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.586900 11411 tablet_bootstrap.cc:492] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.588176 11411 tablet_bootstrap.cc:654] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.613876 11411 tablet_bootstrap.cc:492] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.614041 11411 ts_tablet_manager.cc:1397] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.001s
I20260809 06:39:48.614617 11411 raft_consensus.cc:348] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.614796 11411 raft_consensus.cc:374] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.614852 11411 raft_consensus.cc:729] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.615100 11411 consensus_queue.cc:260] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.615247 11411 raft_consensus.cc:388] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.615314 11411 raft_consensus.cc:482] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.615379 11411 raft_consensus.cc:3037] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.617170 11411 raft_consensus.cc:504] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.617414 11411 leader_election.cc:302] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.617654 11411 leader_election.cc:288] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.617794 11413 raft_consensus.cc:2781] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.617978 11413 raft_consensus.cc:686] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.618137 11411 ts_tablet_manager.cc:1428] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:48.618244 11413 consensus_queue.cc:237] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.621297 11113 catalog_manager.cc:5495] T e3ce67ed6a5d4d76a97d07e7ddc951fc P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.624161 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260809 06:39:48.629763 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260809 06:39:48.637629 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.003s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":248,"dirs.run_wall_time_us":1214,"drs_written":1,"lbm_read_time_us":142,"lbm_reads_lt_1ms":4,"lbm_write_time_us":988,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:48.638576 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.401726
I20260809 06:39:48.640990 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.001s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1234,"drs_written":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4,"lbm_write_time_us":868,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:48.641945 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 601 bytes on disk
I20260809 06:39:48.642518 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.643114 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.387609
I20260809 06:39:48.662022 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.017s sys 0.004s Metrics: {"bytes_written":13537,"cfile_cache_hit":14,"cfile_cache_hit_bytes":7809,"cfile_cache_miss":34,"cfile_cache_miss_bytes":12631,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":458,"dirs.run_cpu_time_us":883,"dirs.run_wall_time_us":4640,"drs_written":1,"lbm_read_time_us":1027,"lbm_reads_lt_1ms":74,"lbm_write_time_us":888,"lbm_writes_lt_1ms":27,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":3766,"rows_written":37}
I20260809 06:39:48.663058 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1735 bytes on disk
I20260809 06:39:48.663709 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.668701 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.020s sys 0.004s Metrics: {"bytes_written":13537,"cfile_cache_hit":13,"cfile_cache_hit_bytes":7815,"cfile_cache_miss":31,"cfile_cache_miss_bytes":12275,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":922,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1072,"drs_written":1,"lbm_read_time_us":1305,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1134,"lbm_writes_lt_1ms":27,"mutex_wait_us":192,"num_input_rowsets":2,"peak_mem_usage":3762,"rows_written":37,"spinlock_wait_cycles":768}
I20260809 06:39:48.669598 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1735 bytes on disk
I20260809 06:39:48.670264 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.672564 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53388:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.683939 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c771b91ccb7d4d5d84b19cb0af85c9ab (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.18 [id=e1c181a962494b0d8b1b579acc7facdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.684504 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c771b91ccb7d4d5d84b19cb0af85c9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.688879 11411 tablet_bootstrap.cc:492] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.690464 11411 tablet_bootstrap.cc:654] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.692214 11411 tablet_bootstrap.cc:492] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.692360 11411 ts_tablet_manager.cc:1397] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.693161 11411 raft_consensus.cc:348] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.693328 11411 raft_consensus.cc:374] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.693482 11411 raft_consensus.cc:729] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.693740 11411 consensus_queue.cc:260] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.693917 11411 raft_consensus.cc:388] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.694020 11411 raft_consensus.cc:482] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.694144 11411 raft_consensus.cc:3037] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.695947 11411 raft_consensus.cc:504] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.696244 11411 leader_election.cc:302] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.696439 11411 leader_election.cc:288] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.696537 11413 raft_consensus.cc:2781] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.696727 11413 raft_consensus.cc:686] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.696903 11411 ts_tablet_manager.cc:1428] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:48.696959 11413 consensus_queue.cc:237] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.699753 11113 catalog_manager.cc:5495] T c771b91ccb7d4d5d84b19cb0af85c9ab P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.727440 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53400:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.738404 11297 tablet_service.cc:1467] Processing CreateTablet for tablet cd56b0dd273c4b1b850dd6a097e281cb (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.19 [id=599b73dd19de4247b903174502e800b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.738988 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd56b0dd273c4b1b850dd6a097e281cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.743856 11411 tablet_bootstrap.cc:492] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.745121 11411 tablet_bootstrap.cc:654] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.747169 11411 tablet_bootstrap.cc:492] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.747334 11411 ts_tablet_manager.cc:1397] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:48.747959 11411 raft_consensus.cc:348] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.748149 11411 raft_consensus.cc:374] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.748255 11411 raft_consensus.cc:729] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.748485 11411 consensus_queue.cc:260] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.748644 11411 raft_consensus.cc:388] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.748780 11411 raft_consensus.cc:482] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.748893 11411 raft_consensus.cc:3037] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.750973 11411 raft_consensus.cc:504] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.751435 11411 leader_election.cc:302] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.751659 11411 leader_election.cc:288] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.751792 11413 raft_consensus.cc:2781] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.751992 11413 raft_consensus.cc:686] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.752230 11411 ts_tablet_manager.cc:1428] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 06:39:48.752249 11413 consensus_queue.cc:237] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.755398 11113 catalog_manager.cc:5495] T cd56b0dd273c4b1b850dd6a097e281cb P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.784288 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53416:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.794358 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 57821b5e358a4c60a35098c8632a5743 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.20 [id=0c046b2adaec4090ae9afe3444f2a937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.794811 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57821b5e358a4c60a35098c8632a5743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.799562 11411 tablet_bootstrap.cc:492] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.800920 11411 tablet_bootstrap.cc:654] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.803042 11411 tablet_bootstrap.cc:492] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.803217 11411 ts_tablet_manager.cc:1397] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.803826 11411 raft_consensus.cc:348] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.804009 11411 raft_consensus.cc:374] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.804140 11411 raft_consensus.cc:729] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.804389 11411 consensus_queue.cc:260] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.804595 11411 raft_consensus.cc:388] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.804700 11411 raft_consensus.cc:482] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.804812 11411 raft_consensus.cc:3037] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.807075 11411 raft_consensus.cc:504] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.807333 11411 leader_election.cc:302] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.807529 11411 leader_election.cc:288] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.807605 11413 raft_consensus.cc:2781] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.807876 11413 raft_consensus.cc:686] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.808132 11413 consensus_queue.cc:237] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.808323 11411 ts_tablet_manager.cc:1428] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:48.811230 11113 catalog_manager.cc:5495] T 57821b5e358a4c60a35098c8632a5743 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.839900 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53424:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.850629 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a94a56b35f004f208eebb40a9e9ef851 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.21 [id=4c96ca0d904c42eda2d45eb1c1f44f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.851014 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a94a56b35f004f208eebb40a9e9ef851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.855792 11411 tablet_bootstrap.cc:492] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.857194 11411 tablet_bootstrap.cc:654] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.859234 11411 tablet_bootstrap.cc:492] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.859407 11411 ts_tablet_manager.cc:1397] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.860034 11411 raft_consensus.cc:348] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.860198 11411 raft_consensus.cc:374] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.860253 11411 raft_consensus.cc:729] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.860409 11411 consensus_queue.cc:260] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.860540 11411 raft_consensus.cc:388] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.860610 11411 raft_consensus.cc:482] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.860673 11411 raft_consensus.cc:3037] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.862371 11411 raft_consensus.cc:504] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.862548 11411 leader_election.cc:302] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.862768 11411 leader_election.cc:288] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.862834 11413 raft_consensus.cc:2781] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.862969 11413 raft_consensus.cc:686] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.863195 11413 consensus_queue.cc:237] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.863891 11411 ts_tablet_manager.cc:1428] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260809 06:39:48.866425 11113 catalog_manager.cc:5495] T a94a56b35f004f208eebb40a9e9ef851 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.904034 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53440:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.914577 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000077
I20260809 06:39:48.914805 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 7c489222ed444d84a1f1800ca1cee1f3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.22 [id=19205879afd14e3f84bb51ed7f74572c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.915289 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c489222ed444d84a1f1800ca1cee1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.921051 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:48.921682 11411 tablet_bootstrap.cc:492] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.923375 11411 tablet_bootstrap.cc:654] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.925711 11411 tablet_bootstrap.cc:492] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.925865 11411 ts_tablet_manager.cc:1397] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260809 06:39:48.926353 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.002s Metrics: {"bytes_written":6922,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":5445,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":944,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:48.926661 11411 raft_consensus.cc:348] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.926853 11411 raft_consensus.cc:374] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.926954 11411 raft_consensus.cc:729] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.927158 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.383336
I20260809 06:39:48.927191 11411 consensus_queue.cc:260] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.927417 11411 raft_consensus.cc:388] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.927512 11411 raft_consensus.cc:482] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.927626 11411 raft_consensus.cc:3037] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.930320 11411 raft_consensus.cc:504] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.930588 11411 leader_election.cc:302] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.930810 11411 leader_election.cc:288] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.930889 11413 raft_consensus.cc:2781] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.931082 11413 raft_consensus.cc:686] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.931368 11411 ts_tablet_manager.cc:1428] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:48.931316 11413 consensus_queue.cc:237] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.933468 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.004s Metrics: {"bytes_written":6922,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1015,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":957,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:48.934306 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 693 bytes on disk
I20260809 06:39:48.934904 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.935940 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.383336
I20260809 06:39:48.936986 11113 catalog_manager.cc:5495] T 7c489222ed444d84a1f1800ca1cee1f3 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:48.957288 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.026s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":9589,"cfile_cache_miss":28,"cfile_cache_miss_bytes":14601,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1432,"dirs.run_cpu_time_us":836,"dirs.run_wall_time_us":5405,"drs_written":1,"lbm_read_time_us":2099,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1077,"lbm_writes_lt_1ms":31,"mutex_wait_us":255,"num_input_rowsets":2,"peak_mem_usage":4764,"reinsert_count":0,"rows_written":47,"thread_start_us":2094,"threads_started":1,"update_count":1}
I20260809 06:39:48.958178 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2141 bytes on disk
I20260809 06:39:48.958714 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.964792 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.024s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":9589,"cfile_cache_miss":28,"cfile_cache_miss_bytes":14601,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":462,"dirs.run_cpu_time_us":567,"dirs.run_wall_time_us":3368,"drs_written":1,"lbm_read_time_us":1331,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1140,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":4764,"reinsert_count":0,"rows_written":47,"update_count":1}
I20260809 06:39:48.965855 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2141 bytes on disk
I20260809 06:39:48.966411 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4}
I20260809 06:39:48.971030 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53442:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:48.981918 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1e832128c8fe4d969db805673567e88a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.23 [id=326b913de5de470689d1873c4fc257cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:48.982287 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e832128c8fe4d969db805673567e88a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:48.987150 11411 tablet_bootstrap.cc:492] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:48.988420 11411 tablet_bootstrap.cc:654] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:48.990689 11411 tablet_bootstrap.cc:492] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:48.990820 11411 ts_tablet_manager.cc:1397] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:48.991326 11411 raft_consensus.cc:348] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.991498 11411 raft_consensus.cc:374] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:48.991552 11411 raft_consensus.cc:729] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:48.991737 11411 consensus_queue.cc:260] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.991860 11411 raft_consensus.cc:388] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:48.991923 11411 raft_consensus.cc:482] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:48.991990 11411 raft_consensus.cc:3037] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:48.993697 11411 raft_consensus.cc:504] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.993949 11411 leader_election.cc:302] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:48.994208 11411 leader_election.cc:288] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:48.994300 11413 raft_consensus.cc:2781] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:48.994495 11413 raft_consensus.cc:686] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:48.994663 11411 ts_tablet_manager.cc:1428] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:48.994788 11413 consensus_queue.cc:237] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:48.998019 11112 catalog_manager.cc:5495] T 1e832128c8fe4d969db805673567e88a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.027732 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53446:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.039537 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 202df1cd83da43d1a4971ce42f212ebb (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.24 [id=985140ec99874512a9834187f27f9287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.040191 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 202df1cd83da43d1a4971ce42f212ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.045637 11411 tablet_bootstrap.cc:492] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.046991 11411 tablet_bootstrap.cc:654] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.048928 11411 tablet_bootstrap.cc:492] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.049052 11411 ts_tablet_manager.cc:1397] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.049726 11411 raft_consensus.cc:348] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.049923 11411 raft_consensus.cc:374] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.050034 11411 raft_consensus.cc:729] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.050279 11411 consensus_queue.cc:260] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.050526 11411 raft_consensus.cc:388] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.050639 11411 raft_consensus.cc:482] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.050760 11411 raft_consensus.cc:3037] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.052614 11411 raft_consensus.cc:504] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.052878 11411 leader_election.cc:302] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.053077 11411 leader_election.cc:288] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.053140 11413 raft_consensus.cc:2781] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.053373 11413 raft_consensus.cc:686] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.053635 11413 consensus_queue.cc:237] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.054236 11411 ts_tablet_manager.cc:1428] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:49.057205 11112 catalog_manager.cc:5495] T 202df1cd83da43d1a4971ce42f212ebb P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.088393 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53450:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.100708 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b7863443398448d389c4d81cfc4970bf (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.25 [id=a40b01b5bc1e4b8fb60c8af8b46f0a1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.101099 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7863443398448d389c4d81cfc4970bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.105621 11411 tablet_bootstrap.cc:492] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.106922 11411 tablet_bootstrap.cc:654] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.109128 11411 tablet_bootstrap.cc:492] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.109304 11411 ts_tablet_manager.cc:1397] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:49.109952 11411 raft_consensus.cc:348] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.110167 11411 raft_consensus.cc:374] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.110284 11411 raft_consensus.cc:729] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.110554 11411 consensus_queue.cc:260] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.110731 11411 raft_consensus.cc:388] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.110850 11411 raft_consensus.cc:482] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.110963 11411 raft_consensus.cc:3037] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.113169 11411 raft_consensus.cc:504] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.113427 11411 leader_election.cc:302] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.113641 11411 leader_election.cc:288] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.113744 11413 raft_consensus.cc:2781] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.113938 11413 raft_consensus.cc:686] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.114127 11411 ts_tablet_manager.cc:1428] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:49.114193 11413 consensus_queue.cc:237] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.117283 11112 catalog_manager.cc:5495] T b7863443398448d389c4d81cfc4970bf P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.143975 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53456:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.154985 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d979c5af837f4967b83a5dd9168b93e9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.26 [id=835a9961723d4b1b8f5f07683799ca96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.155454 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d979c5af837f4967b83a5dd9168b93e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.160264 11411 tablet_bootstrap.cc:492] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.161633 11411 tablet_bootstrap.cc:654] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.163690 11411 tablet_bootstrap.cc:492] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.163844 11411 ts_tablet_manager.cc:1397] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:49.164443 11411 raft_consensus.cc:348] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.164633 11411 raft_consensus.cc:374] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.164740 11411 raft_consensus.cc:729] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.164940 11411 consensus_queue.cc:260] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.165163 11411 raft_consensus.cc:388] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.165252 11411 raft_consensus.cc:482] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.165403 11411 raft_consensus.cc:3037] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.167160 11411 raft_consensus.cc:504] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.167416 11411 leader_election.cc:302] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.167609 11411 leader_election.cc:288] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.167668 11413 raft_consensus.cc:2781] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.167891 11413 raft_consensus.cc:686] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.168092 11411 ts_tablet_manager.cc:1428] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.168094 11413 consensus_queue.cc:237] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.171739 11112 catalog_manager.cc:5495] T d979c5af837f4967b83a5dd9168b93e9 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.198215 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53470:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.209067 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b61d4a99b1a0498a96f3b6850a56ea9c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.27 [id=adc57bed4b854ff4aa3dbe7cabda63f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.209578 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61d4a99b1a0498a96f3b6850a56ea9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.209662 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:49.214345 11411 tablet_bootstrap.cc:492] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.216038 11411 tablet_bootstrap.cc:654] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.217319 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:49.218420 11411 tablet_bootstrap.cc:492] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.218588 11411 ts_tablet_manager.cc:1397] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:49.219477 11411 raft_consensus.cc:348] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.219715 11411 raft_consensus.cc:374] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.219813 11411 raft_consensus.cc:729] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.220129 11411 consensus_queue.cc:260] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.220341 11411 raft_consensus.cc:388] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.220476 11411 raft_consensus.cc:482] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.220577 11411 raft_consensus.cc:3037] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.220760 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.004s sys 0.003s Metrics: {"bytes_written":6921,"cfile_init":1,"dirs.queue_time_us":58,"dirs.run_cpu_time_us":274,"dirs.run_wall_time_us":1489,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":733,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:49.221712 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 662 bytes on disk
I20260809 06:39:49.222333 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.222899 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.352059
I20260809 06:39:49.223191 11411 raft_consensus.cc:504] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.223448 11411 leader_election.cc:302] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.223682 11411 leader_election.cc:288] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.223758 11413 raft_consensus.cc:2781] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.223922 11413 raft_consensus.cc:686] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.224203 11411 ts_tablet_manager.cc:1428] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260809 06:39:49.224176 11413 consensus_queue.cc:237] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.229975 11112 catalog_manager.cc:5495] T b61d4a99b1a0498a96f3b6850a56ea9c P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.230651 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":6921,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":149,"dirs.run_wall_time_us":961,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":981,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:49.231456 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.352059
I20260809 06:39:49.257714 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.025s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11421,"cfile_cache_miss":31,"cfile_cache_miss_bytes":17587,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":769,"dirs.run_cpu_time_us":1032,"dirs.run_wall_time_us":5445,"drs_written":1,"lbm_read_time_us":1384,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1193,"lbm_writes_lt_1ms":31,"mutex_wait_us":40,"num_input_rowsets":2,"peak_mem_usage":6285,"reinsert_count":0,"rows_written":57,"spinlock_wait_cycles":1408,"thread_start_us":3118,"threads_started":1,"update_count":1}
I20260809 06:39:49.259465 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2554 bytes on disk
I20260809 06:39:49.260141 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.260953 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.021s sys 0.002s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":11421,"cfile_cache_miss":31,"cfile_cache_miss_bytes":17587,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":539,"dirs.run_cpu_time_us":872,"dirs.run_wall_time_us":4063,"drs_written":1,"lbm_read_time_us":3098,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":66,"lbm_write_time_us":1321,"lbm_writes_lt_1ms":31,"mutex_wait_us":132,"num_input_rowsets":2,"peak_mem_usage":6285,"reinsert_count":0,"rows_written":57,"spinlock_wait_cycles":3968,"update_count":1}
I20260809 06:39:49.262033 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2554 bytes on disk
I20260809 06:39:49.263136 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.273077 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53484:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.284323 11297 tablet_service.cc:1467] Processing CreateTablet for tablet bb379feced0642de8599ecf768a85a28 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.28 [id=296e17e12644486a8f1871789ec50d6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.284837 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb379feced0642de8599ecf768a85a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.289387 11411 tablet_bootstrap.cc:492] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.290707 11411 tablet_bootstrap.cc:654] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.292605 11411 tablet_bootstrap.cc:492] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.292760 11411 ts_tablet_manager.cc:1397] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:49.293421 11411 raft_consensus.cc:348] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.293610 11411 raft_consensus.cc:374] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.293742 11411 raft_consensus.cc:729] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.293980 11411 consensus_queue.cc:260] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.294175 11411 raft_consensus.cc:388] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.294291 11411 raft_consensus.cc:482] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.294395 11411 raft_consensus.cc:3037] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.296160 11411 raft_consensus.cc:504] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.296433 11411 leader_election.cc:302] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.296653 11411 leader_election.cc:288] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.296777 11413 raft_consensus.cc:2781] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.297010 11413 raft_consensus.cc:686] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.297206 11411 ts_tablet_manager.cc:1428] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260809 06:39:49.297236 11413 consensus_queue.cc:237] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.300503 11112 catalog_manager.cc:5495] T bb379feced0642de8599ecf768a85a28 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.328256 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53486:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.338903 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 81d4195073814036acd9f804d09d59da (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.29 [id=4bac63dcd11849d1b15ad664a6cafd83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.339349 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d4195073814036acd9f804d09d59da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.344244 11411 tablet_bootstrap.cc:492] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.345574 11411 tablet_bootstrap.cc:654] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.347527 11411 tablet_bootstrap.cc:492] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.347697 11411 ts_tablet_manager.cc:1397] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.348284 11411 raft_consensus.cc:348] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.348482 11411 raft_consensus.cc:374] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.348569 11411 raft_consensus.cc:729] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.348834 11411 consensus_queue.cc:260] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.349016 11411 raft_consensus.cc:388] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.349150 11411 raft_consensus.cc:482] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.349262 11411 raft_consensus.cc:3037] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.351191 11411 raft_consensus.cc:504] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.351431 11411 leader_election.cc:302] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.351639 11411 leader_election.cc:288] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.351699 11413 raft_consensus.cc:2781] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.351941 11413 raft_consensus.cc:686] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.352146 11411 ts_tablet_manager.cc:1428] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:49.352211 11413 consensus_queue.cc:237] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.355342 11112 catalog_manager.cc:5495] T 81d4195073814036acd9f804d09d59da P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.383867 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53498:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.395455 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 00353f7c6757426d88ba6b84d26bf6f1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.30 [id=a828850f8a8b457d81e07256df11e4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.395933 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00353f7c6757426d88ba6b84d26bf6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.402545 11411 tablet_bootstrap.cc:492] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.404270 11411 tablet_bootstrap.cc:654] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.406451 11411 tablet_bootstrap.cc:492] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.406621 11411 ts_tablet_manager.cc:1397] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:49.407244 11411 raft_consensus.cc:348] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.407552 11411 raft_consensus.cc:374] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.407691 11411 raft_consensus.cc:729] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.407934 11411 consensus_queue.cc:260] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.408187 11411 raft_consensus.cc:388] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.408303 11411 raft_consensus.cc:482] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.408423 11411 raft_consensus.cc:3037] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.410435 11411 raft_consensus.cc:504] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.410702 11411 leader_election.cc:302] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.410975 11411 leader_election.cc:288] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.411068 11413 raft_consensus.cc:2781] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.411295 11413 raft_consensus.cc:686] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.411516 11413 consensus_queue.cc:237] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.411914 11411 ts_tablet_manager.cc:1428] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:49.415390 11112 catalog_manager.cc:5495] T 00353f7c6757426d88ba6b84d26bf6f1 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.455673 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53502:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.467118 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 6c3a38178e5d4eb4ba8882eb60e3559d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.31 [id=a0891455cbb24dac9aa169c44e91e549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.467594 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c3a38178e5d4eb4ba8882eb60e3559d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.472105 11411 tablet_bootstrap.cc:492] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.473376 11411 tablet_bootstrap.cc:654] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.475361 11411 tablet_bootstrap.cc:492] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.475520 11411 ts_tablet_manager.cc:1397] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:49.476115 11411 raft_consensus.cc:348] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.476295 11411 raft_consensus.cc:374] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.476374 11411 raft_consensus.cc:729] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.476624 11411 consensus_queue.cc:260] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.476789 11411 raft_consensus.cc:388] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.476899 11411 raft_consensus.cc:482] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.477005 11411 raft_consensus.cc:3037] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.479152 11411 raft_consensus.cc:504] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.479382 11411 leader_election.cc:302] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.479592 11411 leader_election.cc:288] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.479660 11413 raft_consensus.cc:2781] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.479801 11413 raft_consensus.cc:686] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.480070 11413 consensus_queue.cc:237] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.480443 11411 ts_tablet_manager.cc:1428] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:49.483299 11112 catalog_manager.cc:5495] T 6c3a38178e5d4eb4ba8882eb60e3559d P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.511308 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:49.514103 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:49.515012 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53506:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.523516 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.006s sys 0.004s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":249,"dirs.run_wall_time_us":1259,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":838,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:49.524428 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.314736
I20260809 06:39:49.526275 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.006s sys 0.005s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":1227,"drs_written":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4,"lbm_write_time_us":911,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:49.527139 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.314736
I20260809 06:39:49.531005 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1290e00d05e7411abcdcf4e1eb1c0ac8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.32 [id=e13266d7018e424a875c9c275e87b34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.531553 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1290e00d05e7411abcdcf4e1eb1c0ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.537922 11411 tablet_bootstrap.cc:492] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.539713 11411 tablet_bootstrap.cc:654] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.549507 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.017s sys 0.004s Metrics: {"bytes_written":19646,"cfile_cache_hit":13,"cfile_cache_hit_bytes":12930,"cfile_cache_miss":31,"cfile_cache_miss_bytes":19766,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":1128,"dirs.run_cpu_time_us":325,"dirs.run_wall_time_us":1745,"drs_written":1,"lbm_read_time_us":1278,"lbm_reads_lt_1ms":67,"lbm_write_time_us":754,"lbm_writes_lt_1ms":27,"mutex_wait_us":203,"num_input_rowsets":2,"peak_mem_usage":7555,"rows_written":65}
I20260809 06:39:49.550158 11411 tablet_bootstrap.cc:492] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.550379 11411 ts_tablet_manager.cc:1397] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260809 06:39:49.550629 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000007
I20260809 06:39:49.550886 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.020s sys 0.003s Metrics: {"bytes_written":19646,"cfile_cache_hit":13,"cfile_cache_hit_bytes":12930,"cfile_cache_miss":31,"cfile_cache_miss_bytes":19766,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":413,"dirs.run_cpu_time_us":681,"dirs.run_wall_time_us":3710,"drs_written":1,"lbm_read_time_us":1306,"lbm_reads_lt_1ms":67,"lbm_write_time_us":730,"lbm_writes_lt_1ms":27,"mutex_wait_us":1,"num_input_rowsets":2,"peak_mem_usage":7555,"rows_written":65}
I20260809 06:39:49.551489 11411 raft_consensus.cc:348] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.551689 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260809 06:39:49.551718 11411 raft_consensus.cc:374] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.551820 11411 raft_consensus.cc:729] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.552099 11411 consensus_queue.cc:260] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.552315 11411 raft_consensus.cc:388] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.552400 11411 raft_consensus.cc:482] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.552529 11411 raft_consensus.cc:3037] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.555148 11411 raft_consensus.cc:504] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.555464 11411 leader_election.cc:302] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.555748 11413 raft_consensus.cc:2781] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.555925 11413 raft_consensus.cc:686] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.556149 11413 consensus_queue.cc:237] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.560987 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.003s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":987,"dirs.run_cpu_time_us":139,"dirs.run_wall_time_us":946,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":845,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:49.561826 11411 leader_election.cc:288] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.561954 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3136 bytes on disk
I20260809 06:39:49.562364 11411 ts_tablet_manager.cc:1428] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260809 06:39:49.566131 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":167,"lbm_reads_lt_1ms":8}
I20260809 06:39:49.566677 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.008s sys 0.004s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":87,"dirs.run_cpu_time_us":142,"dirs.run_wall_time_us":959,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":981,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:49.566998 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.188757
I20260809 06:39:49.567494 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3136 bytes on disk
I20260809 06:39:49.568252 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":168,"lbm_reads_lt_1ms":8}
I20260809 06:39:49.568987 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.188757
I20260809 06:39:49.568810 11112 catalog_manager.cc:5495] T 1290e00d05e7411abcdcf4e1eb1c0ac8 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.591859 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.024s sys 0.000s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":13275,"cfile_cache_miss":28,"cfile_cache_miss_bytes":19695,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":756,"dirs.run_cpu_time_us":908,"dirs.run_wall_time_us":4786,"drs_written":1,"lbm_read_time_us":1167,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1067,"lbm_writes_lt_1ms":31,"mutex_wait_us":80,"num_input_rowsets":2,"peak_mem_usage":8518,"reinsert_count":0,"rows_written":67,"thread_start_us":357,"threads_started":1,"update_count":1}
I20260809 06:39:49.592727 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260809 06:39:49.593987 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.022s sys 0.000s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":13275,"cfile_cache_miss":28,"cfile_cache_miss_bytes":19695,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":791,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":963,"drs_written":1,"lbm_read_time_us":929,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1181,"lbm_writes_lt_1ms":31,"mutex_wait_us":86,"num_input_rowsets":2,"peak_mem_usage":8518,"reinsert_count":0,"rows_written":67,"spinlock_wait_cycles":2176,"update_count":1}
I20260809 06:39:49.595032 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2971 bytes on disk
I20260809 06:39:49.595552 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.596153 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260809 06:39:49.597047 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.000s sys 0.001s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":166,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:49.597540 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2971 bytes on disk
I20260809 06:39:49.597981 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.598409 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:49.602144 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":140,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:49.602671 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:49.604120 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":461,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":159,"lbm_read_time_us":91,"lbm_reads_lt_1ms":8,"lbm_write_time_us":111,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 06:39:49.609157 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":461,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":48,"lbm_read_time_us":145,"lbm_reads_lt_1ms":8,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 06:39:49.622761 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53514:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.633572 11297 tablet_service.cc:1467] Processing CreateTablet for tablet bda6ba2e74c445779149c25d2451e5d0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.33 [id=8b856682e3524437a44c2f56ee4012a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.634068 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bda6ba2e74c445779149c25d2451e5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.638656 11411 tablet_bootstrap.cc:492] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.640137 11411 tablet_bootstrap.cc:654] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.642298 11411 tablet_bootstrap.cc:492] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.642468 11411 ts_tablet_manager.cc:1397] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:49.643080 11411 raft_consensus.cc:348] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.643277 11411 raft_consensus.cc:374] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.643364 11411 raft_consensus.cc:729] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.643664 11411 consensus_queue.cc:260] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.643826 11411 raft_consensus.cc:388] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.643942 11411 raft_consensus.cc:482] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.644059 11411 raft_consensus.cc:3037] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.645903 11411 raft_consensus.cc:504] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.646126 11411 leader_election.cc:302] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.646366 11411 leader_election.cc:288] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.646489 11413 raft_consensus.cc:2781] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.646710 11413 raft_consensus.cc:686] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.646862 11411 ts_tablet_manager.cc:1428] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:49.647032 11413 consensus_queue.cc:237] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
W20260809 06:39:49.648248 11382 tablet.cc:2369] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:39:49.650916 11112 catalog_manager.cc:5495] T bda6ba2e74c445779149c25d2451e5d0 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.679893 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53526:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.692042 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 9432e7ebac504761b4c225c6e1d1cef9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.34 [id=a71913c4a71b45a5a72075a92e11fdef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.692482 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9432e7ebac504761b4c225c6e1d1cef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.696825 11411 tablet_bootstrap.cc:492] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.698351 11411 tablet_bootstrap.cc:654] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.700186 11411 tablet_bootstrap.cc:492] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.700348 11411 ts_tablet_manager.cc:1397] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:49.700925 11411 raft_consensus.cc:348] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.701103 11411 raft_consensus.cc:374] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.701263 11411 raft_consensus.cc:729] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.701562 11411 consensus_queue.cc:260] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.701766 11411 raft_consensus.cc:388] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.701870 11411 raft_consensus.cc:482] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.701984 11411 raft_consensus.cc:3037] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.703665 11411 raft_consensus.cc:504] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.703917 11411 leader_election.cc:302] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.704119 11411 leader_election.cc:288] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.704190 11413 raft_consensus.cc:2781] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.704442 11413 raft_consensus.cc:686] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.704618 11411 ts_tablet_manager.cc:1428] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.704728 11413 consensus_queue.cc:237] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.708154 11112 catalog_manager.cc:5495] T 9432e7ebac504761b4c225c6e1d1cef9 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.734825 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53540:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.745848 11297 tablet_service.cc:1467] Processing CreateTablet for tablet de30054ee6cd4a0db869f3fe7ec15f10 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.35 [id=b349bc319910489e8f8bbd1bfd1f42b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.746333 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de30054ee6cd4a0db869f3fe7ec15f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.750679 11411 tablet_bootstrap.cc:492] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.751940 11411 tablet_bootstrap.cc:654] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.754102 11411 tablet_bootstrap.cc:492] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.754340 11411 ts_tablet_manager.cc:1397] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.755012 11411 raft_consensus.cc:348] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.755192 11411 raft_consensus.cc:374] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.755271 11411 raft_consensus.cc:729] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.755514 11411 consensus_queue.cc:260] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.755716 11411 raft_consensus.cc:388] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.755793 11411 raft_consensus.cc:482] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.755860 11411 raft_consensus.cc:3037] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.757618 11411 raft_consensus.cc:504] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.757809 11411 leader_election.cc:302] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.758006 11411 leader_election.cc:288] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.758132 11413 raft_consensus.cc:2781] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.758327 11413 raft_consensus.cc:686] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.758584 11411 ts_tablet_manager.cc:1428] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 06:39:49.758580 11413 consensus_queue.cc:237] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.762912 11113 catalog_manager.cc:5495] T de30054ee6cd4a0db869f3fe7ec15f10 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.790009 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53548:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.799787 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b0869f0bf2854433a8a30e113aa65c22 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.36 [id=c6517574f1214f8ab338b89d23e31672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.800212 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0869f0bf2854433a8a30e113aa65c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.805135 11411 tablet_bootstrap.cc:492] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.806481 11411 tablet_bootstrap.cc:654] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.808588 11411 tablet_bootstrap.cc:492] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.808699 11411 ts_tablet_manager.cc:1397] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.809262 11411 raft_consensus.cc:348] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.809473 11411 raft_consensus.cc:374] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.809551 11411 raft_consensus.cc:729] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.809722 11411 consensus_queue.cc:260] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.809890 11411 raft_consensus.cc:388] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.809974 11411 raft_consensus.cc:482] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.810163 11411 raft_consensus.cc:3037] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.812103 11411 raft_consensus.cc:504] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.812357 11411 leader_election.cc:302] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.812543 11411 leader_election.cc:288] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.812672 11413 raft_consensus.cc:2781] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.812870 11413 raft_consensus.cc:686] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.813033 11411 ts_tablet_manager.cc:1428] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:49.813102 11413 consensus_queue.cc:237] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.816392 11113 catalog_manager.cc:5495] T b0869f0bf2854433a8a30e113aa65c22 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.844580 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53558:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.854871 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d08b5961b7e541d6b553e4beba4316ce (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.37 [id=27524bd0871c457e8a0c4c7e33a7af83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.855067 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260809 06:39:49.855365 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d08b5961b7e541d6b553e4beba4316ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.860026 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:49.861768 11411 tablet_bootstrap.cc:492] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.863576 11411 tablet_bootstrap.cc:654] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.866113 11411 tablet_bootstrap.cc:492] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.866285 11411 ts_tablet_manager.cc:1397] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 06:39:49.867141 11411 raft_consensus.cc:348] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.867357 11411 raft_consensus.cc:374] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.867452 11411 raft_consensus.cc:729] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.867211 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.006s sys 0.004s Metrics: {"bytes_written":6921,"cfile_init":1,"dirs.queue_time_us":77,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1226,"drs_written":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4,"lbm_write_time_us":981,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:49.867738 11411 consensus_queue.cc:260] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.867940 11411 raft_consensus.cc:388] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.868050 11411 raft_consensus.cc:482] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.868149 11411 raft_consensus.cc:3037] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.868239 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 668 bytes on disk
I20260809 06:39:49.868858 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.869606 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.318924
I20260809 06:39:49.870841 11411 raft_consensus.cc:504] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.871083 11411 leader_election.cc:302] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.871315 11411 leader_election.cc:288] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.871416 11413 raft_consensus.cc:2781] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.871579 11413 raft_consensus.cc:686] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.871742 11411 ts_tablet_manager.cc:1428] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:49.871804 11413 consensus_queue.cc:237] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.873238 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.006s Metrics: {"bytes_written":6921,"cfile_init":1,"dirs.queue_time_us":74,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1162,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1256,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:49.873966 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.318924
I20260809 06:39:49.880776 11112 catalog_manager.cc:5495] T d08b5961b7e541d6b553e4beba4316ce P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.900655 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.020s sys 0.005s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15101,"cfile_cache_miss":31,"cfile_cache_miss_bytes":23077,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":441,"dirs.run_cpu_time_us":1023,"dirs.run_wall_time_us":7913,"drs_written":1,"lbm_read_time_us":1498,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1134,"lbm_writes_lt_1ms":31,"mutex_wait_us":19,"num_input_rowsets":2,"peak_mem_usage":9027,"reinsert_count":0,"rows_written":77,"update_count":1}
I20260809 06:39:49.901024 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.023s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":15101,"cfile_cache_miss":31,"cfile_cache_miss_bytes":23077,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":332,"dirs.run_cpu_time_us":301,"dirs.run_wall_time_us":1134,"drs_written":1,"lbm_read_time_us":1392,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1076,"lbm_writes_lt_1ms":31,"mutex_wait_us":44,"num_input_rowsets":2,"peak_mem_usage":9027,"reinsert_count":0,"rows_written":77,"spinlock_wait_cycles":3968,"update_count":1}
I20260809 06:39:49.901865 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3392 bytes on disk
I20260809 06:39:49.901957 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3392 bytes on disk
I20260809 06:39:49.902415 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.903473 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:49.915887 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53564:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.927222 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 873990891bc14f50b6b5871192fdb97f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.38 [id=1e15ff7becf848f5826c060aebaa5d1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.927722 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 873990891bc14f50b6b5871192fdb97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.932668 11411 tablet_bootstrap.cc:492] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.933940 11411 tablet_bootstrap.cc:654] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.936197 11411 tablet_bootstrap.cc:492] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.936355 11411 ts_tablet_manager.cc:1397] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:49.936977 11411 raft_consensus.cc:348] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.937153 11411 raft_consensus.cc:374] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.937266 11411 raft_consensus.cc:729] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.937508 11411 consensus_queue.cc:260] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.937704 11411 raft_consensus.cc:388] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.937800 11411 raft_consensus.cc:482] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.937922 11411 raft_consensus.cc:3037] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.939968 11411 raft_consensus.cc:504] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.940259 11411 leader_election.cc:302] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.940460 11411 leader_election.cc:288] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.940536 11413 raft_consensus.cc:2781] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.940785 11413 raft_consensus.cc:686] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.940997 11413 consensus_queue.cc:237] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.941319 11411 ts_tablet_manager.cc:1428] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:49.944746 11113 catalog_manager.cc:5495] T 873990891bc14f50b6b5871192fdb97f P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:49.972927 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53570:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:49.983470 11297 tablet_service.cc:1467] Processing CreateTablet for tablet f0975c6635f640bcb20c0e4ff8785b7b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.39 [id=d59583bf82c14d62bcdc28a5f373bef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:49.983951 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0975c6635f640bcb20c0e4ff8785b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:49.988658 11411 tablet_bootstrap.cc:492] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:49.990124 11411 tablet_bootstrap.cc:654] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:49.992087 11411 tablet_bootstrap.cc:492] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:49.992259 11411 ts_tablet_manager.cc:1397] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:49.992836 11411 raft_consensus.cc:348] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.993024 11411 raft_consensus.cc:374] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:49.993125 11411 raft_consensus.cc:729] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:49.993424 11411 consensus_queue.cc:260] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.993611 11411 raft_consensus.cc:388] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:49.993733 11411 raft_consensus.cc:482] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:49.993851 11411 raft_consensus.cc:3037] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:49.995496 11411 raft_consensus.cc:504] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.995759 11411 leader_election.cc:302] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:49.995972 11411 leader_election.cc:288] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:49.996031 11413 raft_consensus.cc:2781] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:49.996273 11413 raft_consensus.cc:686] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:49.996544 11413 consensus_queue.cc:237] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:49.996803 11411 ts_tablet_manager.cc:1428] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:50.000017 11112 catalog_manager.cc:5495] T f0975c6635f640bcb20c0e4ff8785b7b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.029171 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53578:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.040534 11297 tablet_service.cc:1467] Processing CreateTablet for tablet f6f0676bb82a4f7fb8bc82cd3b9b0490 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.40 [id=b4bbfdb32f584fe4b9035da2203bdf6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.040997 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6f0676bb82a4f7fb8bc82cd3b9b0490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.047312 11411 tablet_bootstrap.cc:492] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.048605 11411 tablet_bootstrap.cc:654] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.050491 11411 tablet_bootstrap.cc:492] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.050643 11411 ts_tablet_manager.cc:1397] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:39:50.051347 11411 raft_consensus.cc:348] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.051538 11411 raft_consensus.cc:374] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.051684 11411 raft_consensus.cc:729] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.051923 11411 consensus_queue.cc:260] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.052095 11411 raft_consensus.cc:388] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.052218 11411 raft_consensus.cc:482] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.052343 11411 raft_consensus.cc:3037] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.054356 11411 raft_consensus.cc:504] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.054679 11411 leader_election.cc:302] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.054878 11411 leader_election.cc:288] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.054955 11413 raft_consensus.cc:2781] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.055114 11413 raft_consensus.cc:686] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.055351 11413 consensus_queue.cc:237] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.055522 11411 ts_tablet_manager.cc:1428] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:50.059466 11112 catalog_manager.cc:5495] T f6f0676bb82a4f7fb8bc82cd3b9b0490 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.085836 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53592:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.096751 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc0a7f5140c4775865badef5bdbd911 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.41 [id=9b18955ee1c14cf9a7fa5d6f32893d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.097191 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc0a7f5140c4775865badef5bdbd911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.102083 11411 tablet_bootstrap.cc:492] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.103929 11411 tablet_bootstrap.cc:654] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.106228 11411 tablet_bootstrap.cc:492] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.106439 11411 ts_tablet_manager.cc:1397] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:50.107143 11411 raft_consensus.cc:348] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.107324 11411 raft_consensus.cc:374] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.107383 11411 raft_consensus.cc:729] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.107620 11411 consensus_queue.cc:260] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.107776 11411 raft_consensus.cc:388] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.107851 11411 raft_consensus.cc:482] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.107937 11411 raft_consensus.cc:3037] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.109963 11411 raft_consensus.cc:504] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.110152 11411 leader_election.cc:302] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.110354 11411 leader_election.cc:288] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.110417 11413 raft_consensus.cc:2781] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.110550 11413 raft_consensus.cc:686] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.110780 11413 consensus_queue.cc:237] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.110939 11411 ts_tablet_manager.cc:1428] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:50.114220 11113 catalog_manager.cc:5495] T 4fc0a7f5140c4775865badef5bdbd911 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.143642 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53604:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.153254 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:50.154322 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:50.161820 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 795aa0d706cb4f4a863f0929bedd1041 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.42 [id=9c36450d243548968731692244c1c9e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.162328 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 795aa0d706cb4f4a863f0929bedd1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.165577 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":456,"dirs.run_cpu_time_us":182,"dirs.run_wall_time_us":1337,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":967,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":10,"update_count":1}
I20260809 06:39:50.166441 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.317610
I20260809 06:39:50.167407 11411 tablet_bootstrap.cc:492] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.169165 11411 tablet_bootstrap.cc:654] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.170221 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.008s sys 0.003s Metrics: {"bytes_written":6921,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":214,"dirs.run_wall_time_us":1121,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":925,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:50.171228 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.309194
I20260809 06:39:50.171802 11411 tablet_bootstrap.cc:492] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.171962 11411 ts_tablet_manager.cc:1397] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:50.172749 11411 raft_consensus.cc:348] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.173002 11411 raft_consensus.cc:374] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.173115 11411 raft_consensus.cc:729] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.173395 11411 consensus_queue.cc:260] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.173631 11411 raft_consensus.cc:388] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.173727 11411 raft_consensus.cc:482] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.173846 11411 raft_consensus.cc:3037] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.176306 11411 raft_consensus.cc:504] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.176592 11411 leader_election.cc:302] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.176811 11411 leader_election.cc:288] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.176909 11413 raft_consensus.cc:2781] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.177098 11413 raft_consensus.cc:686] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.177307 11411 ts_tablet_manager.cc:1428] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:50.177320 11413 consensus_queue.cc:237] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.181706 11113 catalog_manager.cc:5495] T 795aa0d706cb4f4a863f0929bedd1041 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.194075 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.020s sys 0.006s Metrics: {"bytes_written":24393,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16868,"cfile_cache_miss":34,"cfile_cache_miss_bytes":25896,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":2884,"dirs.run_cpu_time_us":1009,"dirs.run_wall_time_us":6624,"drs_written":1,"lbm_read_time_us":1467,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1018,"lbm_writes_lt_1ms":27,"mutex_wait_us":47,"num_input_rowsets":2,"peak_mem_usage":10298,"rows_written":87,"spinlock_wait_cycles":9600,"thread_start_us":1911,"threads_started":1}
I20260809 06:39:50.197628 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3802 bytes on disk
I20260809 06:39:50.198189 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.198756 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260809 06:39:50.202611 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.026s sys 0.001s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":16946,"cfile_cache_miss":31,"cfile_cache_miss_bytes":25678,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":909,"dirs.run_cpu_time_us":142,"dirs.run_wall_time_us":1121,"drs_written":1,"lbm_read_time_us":1358,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1159,"lbm_writes_lt_1ms":31,"mutex_wait_us":265,"num_input_rowsets":2,"peak_mem_usage":10298,"reinsert_count":0,"rows_written":87,"spinlock_wait_cycles":2432,"update_count":1}
I20260809 06:39:50.203516 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3802 bytes on disk
I20260809 06:39:50.204071 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.204072 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.001s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":132,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:50.216328 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53606:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.227381 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d2907ab24184497084ccbc07e520b865 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.43 [id=ccc8343583e24ce0923939c8a692aabe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.227860 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2907ab24184497084ccbc07e520b865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.232424 11411 tablet_bootstrap.cc:492] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.233919 11411 tablet_bootstrap.cc:654] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.235939 11411 tablet_bootstrap.cc:492] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.236151 11411 ts_tablet_manager.cc:1397] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:50.236768 11411 raft_consensus.cc:348] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.236967 11411 raft_consensus.cc:374] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.237072 11411 raft_consensus.cc:729] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.237377 11411 consensus_queue.cc:260] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.237564 11411 raft_consensus.cc:388] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.237672 11411 raft_consensus.cc:482] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.237776 11411 raft_consensus.cc:3037] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.239696 11411 raft_consensus.cc:504] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.239943 11411 leader_election.cc:302] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.240149 11411 leader_election.cc:288] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.240271 11413 raft_consensus.cc:2781] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.240448 11413 raft_consensus.cc:686] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.240741 11411 ts_tablet_manager.cc:1428] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260809 06:39:50.240686 11413 consensus_queue.cc:237] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.244540 11112 catalog_manager.cc:5495] T d2907ab24184497084ccbc07e520b865 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.271397 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53622:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.282246 11297 tablet_service.cc:1467] Processing CreateTablet for tablet bb17c1f2214b4ed3bcaac23c07347810 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.44 [id=31383f022cb648cc80ec69999bbc2421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.282693 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb17c1f2214b4ed3bcaac23c07347810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.286945 11411 tablet_bootstrap.cc:492] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.288126 11411 tablet_bootstrap.cc:654] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.290603 11411 tablet_bootstrap.cc:492] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.290778 11411 ts_tablet_manager.cc:1397] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:50.291392 11411 raft_consensus.cc:348] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.291582 11411 raft_consensus.cc:374] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.291697 11411 raft_consensus.cc:729] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.291924 11411 consensus_queue.cc:260] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.292111 11411 raft_consensus.cc:388] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.292205 11411 raft_consensus.cc:482] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.292313 11411 raft_consensus.cc:3037] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.294389 11411 raft_consensus.cc:504] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.294664 11411 leader_election.cc:302] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.294884 11411 leader_election.cc:288] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.294986 11413 raft_consensus.cc:2781] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.295264 11413 raft_consensus.cc:686] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.295362 11411 ts_tablet_manager.cc:1428] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:50.295480 11413 consensus_queue.cc:237] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.299292 11112 catalog_manager.cc:5495] T bb17c1f2214b4ed3bcaac23c07347810 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.326537 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53630:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.336759 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 8f4c3c6274c64d18ba1821fa00cbec52 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.45 [id=88a7df27a583490fb7d9a26a3e71baca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.337145 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f4c3c6274c64d18ba1821fa00cbec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.341866 11411 tablet_bootstrap.cc:492] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.343245 11411 tablet_bootstrap.cc:654] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.345461 11411 tablet_bootstrap.cc:492] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.345597 11411 ts_tablet_manager.cc:1397] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:50.346153 11411 raft_consensus.cc:348] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.346310 11411 raft_consensus.cc:374] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.346374 11411 raft_consensus.cc:729] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.346585 11411 consensus_queue.cc:260] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.346835 11413 raft_consensus.cc:482] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:50.346988 11413 raft_consensus.cc:504] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.346755 11411 raft_consensus.cc:388] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.347277 11413 leader_election.cc:302] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.347363 11411 raft_consensus.cc:482] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.347466 11411 raft_consensus.cc:3037] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.347502 11413 leader_election.cc:288] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 06:39:50.349472 11411 raft_consensus.cc:504] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.349712 11411 leader_election.cc:302] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.349803 11415 raft_consensus.cc:2741] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 06:39:50.349944 11411 leader_election.cc:288] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.350069 11413 raft_consensus.cc:2781] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.350268 11413 raft_consensus.cc:686] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.350488 11413 consensus_queue.cc:237] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.350775 11411 ts_tablet_manager.cc:1428] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:50.354701 11112 catalog_manager.cc:5495] T 8f4c3c6274c64d18ba1821fa00cbec52 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.382265 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53644:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.394994 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 6b616f9167674c9289d935da5b30205e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.46 [id=cfe01274548446758288a081bdd8b124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.395500 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b616f9167674c9289d935da5b30205e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.400453 11411 tablet_bootstrap.cc:492] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.402254 11411 tablet_bootstrap.cc:654] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.404570 11411 tablet_bootstrap.cc:492] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.404745 11411 ts_tablet_manager.cc:1397] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:50.405565 11411 raft_consensus.cc:348] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.405777 11411 raft_consensus.cc:374] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.405864 11411 raft_consensus.cc:729] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.406093 11411 consensus_queue.cc:260] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.406301 11411 raft_consensus.cc:388] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.406397 11411 raft_consensus.cc:482] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.406517 11411 raft_consensus.cc:3037] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.409205 11411 raft_consensus.cc:504] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.409484 11411 leader_election.cc:302] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.409686 11411 leader_election.cc:288] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.409788 11413 raft_consensus.cc:2781] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.410003 11413 raft_consensus.cc:686] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.410168 11411 ts_tablet_manager.cc:1428] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 06:39:50.410306 11413 consensus_queue.cc:237] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.415711 11112 catalog_manager.cc:5495] T 6b616f9167674c9289d935da5b30205e P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.450040 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53654:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.454932 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:50.455054 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:50.468326 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.000s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":1217,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":764,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:50.469198 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 597 bytes on disk
I20260809 06:39:50.469944 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.469962 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d0f927d1ef7e4ab580cc1b391e4d7498 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.47 [id=cd84112694494eeba81d5917fbb4702e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.470412 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f927d1ef7e4ab580cc1b391e4d7498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.470752 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.284349
I20260809 06:39:50.472189 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.009s sys 0.000s Metrics: {"bytes_written":6808,"cfile_init":1,"dirs.queue_time_us":75,"dirs.run_cpu_time_us":242,"dirs.run_wall_time_us":971,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":823,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":10}
I20260809 06:39:50.473083 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.293696
I20260809 06:39:50.475658 11411 tablet_bootstrap.cc:492] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.477355 11411 tablet_bootstrap.cc:654] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.479701 11411 tablet_bootstrap.cc:492] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.479880 11411 ts_tablet_manager.cc:1397] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:50.480470 11411 raft_consensus.cc:348] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.480674 11411 raft_consensus.cc:374] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.480773 11411 raft_consensus.cc:729] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.481029 11411 consensus_queue.cc:260] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.481231 11411 raft_consensus.cc:388] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.481364 11411 raft_consensus.cc:482] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.481470 11411 raft_consensus.cc:3037] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.483384 11411 raft_consensus.cc:504] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.483632 11411 leader_election.cc:302] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.483868 11411 leader_election.cc:288] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.483917 11413 raft_consensus.cc:2781] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.484090 11413 raft_consensus.cc:686] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.484359 11411 ts_tablet_manager.cc:1428] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:50.484325 11413 consensus_queue.cc:237] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.488561 11112 catalog_manager.cc:5495] T d0f927d1ef7e4ab580cc1b391e4d7498 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.507500 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.020s sys 0.005s Metrics: {"bytes_written":26174,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18445,"cfile_cache_miss":31,"cfile_cache_miss_bytes":27805,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":399,"dirs.run_cpu_time_us":182,"dirs.run_wall_time_us":1113,"drs_written":1,"lbm_read_time_us":1386,"lbm_reads_lt_1ms":63,"lbm_write_time_us":876,"lbm_writes_lt_1ms":27,"mutex_wait_us":23,"num_input_rowsets":2,"peak_mem_usage":11568,"rows_written":95,"spinlock_wait_cycles":768}
I20260809 06:39:50.508390 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:50.510139 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.016s sys 0.011s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":18669,"cfile_cache_miss":31,"cfile_cache_miss_bytes":28127,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":490,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1546,"drs_written":1,"lbm_read_time_us":1592,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1187,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":11568,"reinsert_count":0,"rows_written":97,"update_count":1}
I20260809 06:39:50.511188 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4211 bytes on disk
I20260809 06:39:50.511842 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.519117 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"bytes_written":5193,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1078,"drs_written":1,"lbm_read_time_us":46,"lbm_reads_lt_1ms":4,"lbm_write_time_us":739,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:50.520241 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4502 bytes on disk
I20260809 06:39:50.521212 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":221,"lbm_reads_lt_1ms":8}
I20260809 06:39:50.521994 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.209433
I20260809 06:39:50.532577 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53670:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.546191 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc9964641e54e56b70b95fa6b41ded4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.48 [id=50a083705e014404b58f391f02f29052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.545957 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.023s sys 0.000s Metrics: {"bytes_written":26617,"cfile_cache_hit":12,"cfile_cache_hit_bytes":18828,"cfile_cache_miss":28,"cfile_cache_miss_bytes":28008,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":764,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":1315,"drs_written":1,"lbm_read_time_us":1204,"lbm_reads_lt_1ms":56,"lbm_write_time_us":926,"lbm_writes_lt_1ms":27,"mutex_wait_us":42,"num_input_rowsets":2,"peak_mem_usage":12531,"rows_written":97,"spinlock_wait_cycles":896}
I20260809 06:39:50.546713 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc9964641e54e56b70b95fa6b41ded4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.546945 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260809 06:39:50.551107 11411 tablet_bootstrap.cc:492] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.552752 11411 tablet_bootstrap.cc:654] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.555377 11411 tablet_bootstrap.cc:492] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.555552 11411 ts_tablet_manager.cc:1397] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 06:39:50.556386 11411 raft_consensus.cc:348] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.556622 11411 raft_consensus.cc:374] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.556725 11411 raft_consensus.cc:729] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.556974 11411 consensus_queue.cc:260] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.557190 11413 raft_consensus.cc:482] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:50.557391 11413 raft_consensus.cc:504] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.557192 11411 raft_consensus.cc:388] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.557670 11413 leader_election.cc:302] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.557715 11411 raft_consensus.cc:482] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.557554 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1047,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":637,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:50.557859 11411 raft_consensus.cc:3037] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.557960 11413 leader_election.cc:288] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 06:39:50.558344 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4457 bytes on disk
I20260809 06:39:50.558975 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":107,"lbm_reads_lt_1ms":8}
I20260809 06:39:50.559717 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.209370
I20260809 06:39:50.559986 11411 raft_consensus.cc:504] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.560292 11411 leader_election.cc:302] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.560312 11415 raft_consensus.cc:2741] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 06:39:50.560530 11411 leader_election.cc:288] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.560592 11413 raft_consensus.cc:2781] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.560786 11413 raft_consensus.cc:686] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.561005 11413 consensus_queue.cc:237] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.561793 11411 ts_tablet_manager.cc:1428] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 06:39:50.566310 11113 catalog_manager.cc:5495] T 9bc9964641e54e56b70b95fa6b41ded4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.584426 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.021s sys 0.003s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":19163,"cfile_cache_miss":28,"cfile_cache_miss_bytes":28263,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":763,"dirs.run_cpu_time_us":234,"dirs.run_wall_time_us":1172,"drs_written":1,"lbm_read_time_us":1174,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1262,"lbm_writes_lt_1ms":31,"mutex_wait_us":20,"num_input_rowsets":2,"peak_mem_usage":12785,"reinsert_count":0,"rows_written":99,"spinlock_wait_cycles":20992,"update_count":1}
I20260809 06:39:50.585788 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4292 bytes on disk
I20260809 06:39:50.586412 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.586992 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260809 06:39:50.591166 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.001s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":108,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:50.593590 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:50.599328 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":461,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":61,"lbm_read_time_us":152,"lbm_reads_lt_1ms":8,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 06:39:50.610323 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53678:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.621714 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 84de4590a0d54e97b472119d2f9bd76f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.49 [id=537d64a9196d4acfb06220d2c93f0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.622159 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84de4590a0d54e97b472119d2f9bd76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.626797 11411 tablet_bootstrap.cc:492] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.628217 11411 tablet_bootstrap.cc:654] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.645874 11411 tablet_bootstrap.cc:492] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.646054 11411 ts_tablet_manager.cc:1397] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260809 06:39:50.646780 11411 raft_consensus.cc:348] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.646968 11411 raft_consensus.cc:374] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.647027 11411 raft_consensus.cc:729] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.647200 11411 consensus_queue.cc:260] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.647329 11411 raft_consensus.cc:388] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.647418 11411 raft_consensus.cc:482] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.647491 11411 raft_consensus.cc:3037] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.649411 11411 raft_consensus.cc:504] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.649657 11411 leader_election.cc:302] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.649875 11411 leader_election.cc:288] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.649991 11413 raft_consensus.cc:2781] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.650225 11413 raft_consensus.cc:686] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.650681 11411 ts_tablet_manager.cc:1428] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:50.650635 11413 consensus_queue.cc:237] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.655224 11113 catalog_manager.cc:5495] T 84de4590a0d54e97b472119d2f9bd76f P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.684628 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53690:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.696816 11297 tablet_service.cc:1467] Processing CreateTablet for tablet e79392451d864918a2d2822da5cf4c3f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.50 [id=3f19b445fff24884ad0fd8699d80cd5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.697304 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79392451d864918a2d2822da5cf4c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.702131 11411 tablet_bootstrap.cc:492] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.703609 11411 tablet_bootstrap.cc:654] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.705602 11411 tablet_bootstrap.cc:492] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.705770 11411 ts_tablet_manager.cc:1397] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:50.706378 11411 raft_consensus.cc:348] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.706586 11411 raft_consensus.cc:374] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.706713 11411 raft_consensus.cc:729] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.707010 11411 consensus_queue.cc:260] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.707271 11411 raft_consensus.cc:388] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.707355 11411 raft_consensus.cc:482] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.707482 11411 raft_consensus.cc:3037] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.709272 11411 raft_consensus.cc:504] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.709573 11411 leader_election.cc:302] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.709772 11411 leader_election.cc:288] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.709877 11413 raft_consensus.cc:2781] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.710091 11413 raft_consensus.cc:686] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.710306 11413 consensus_queue.cc:237] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.710428 11411 ts_tablet_manager.cc:1428] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:50.714419 11113 catalog_manager.cc:5495] T e79392451d864918a2d2822da5cf4c3f P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.742791 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53700:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.753139 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 65ca89a7664d47ce8fa8c0cf97fae1c6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.51 [id=6060cd409ecc439aa981d25c5b59d4f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.753631 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ca89a7664d47ce8fa8c0cf97fae1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.758296 11411 tablet_bootstrap.cc:492] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.759634 11411 tablet_bootstrap.cc:654] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.761785 11411 tablet_bootstrap.cc:492] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.761960 11411 ts_tablet_manager.cc:1397] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:50.762579 11411 raft_consensus.cc:348] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.762686 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:50.762786 11411 raft_consensus.cc:374] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.762920 11411 raft_consensus.cc:729] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.763176 11411 consensus_queue.cc:260] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.763341 11411 raft_consensus.cc:388] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.763458 11411 raft_consensus.cc:482] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.763569 11411 raft_consensus.cc:3037] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.765648 11411 raft_consensus.cc:504] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.765875 11411 leader_election.cc:302] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.766085 11411 leader_election.cc:288] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.766189 11413 raft_consensus.cc:2781] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.766376 11413 raft_consensus.cc:686] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.766655 11413 consensus_queue.cc:237] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.766928 11411 ts_tablet_manager.cc:1428] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260809 06:39:50.771602 11112 catalog_manager.cc:5495] T 65ca89a7664d47ce8fa8c0cf97fae1c6 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.774417 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.004s Metrics: {"bytes_written":6454,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":207,"dirs.run_wall_time_us":1753,"drs_written":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4,"lbm_write_time_us":863,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:50.775703 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.276789
I20260809 06:39:50.802553 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53702:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.804947 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.019s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20253,"cfile_cache_miss":31,"cfile_cache_miss_bytes":30481,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1191,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":1163,"drs_written":1,"lbm_read_time_us":1119,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1157,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":12838,"reinsert_count":0,"rows_written":105,"update_count":1}
I20260809 06:39:50.806017 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4544 bytes on disk
I20260809 06:39:50.806587 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.814065 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 2c80d691ee644eabb2479a9b72cc4057 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.52 [id=c84359312e854d88a2536ded6603e6e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.814589 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c80d691ee644eabb2479a9b72cc4057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.819191 11411 tablet_bootstrap.cc:492] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.820459 11411 tablet_bootstrap.cc:654] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.822654 11411 tablet_bootstrap.cc:492] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.822808 11411 ts_tablet_manager.cc:1397] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:50.823390 11411 raft_consensus.cc:348] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.823592 11411 raft_consensus.cc:374] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.823730 11411 raft_consensus.cc:729] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.823956 11411 consensus_queue.cc:260] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.824150 11411 raft_consensus.cc:388] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.824239 11411 raft_consensus.cc:482] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.824370 11411 raft_consensus.cc:3037] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.826504 11411 raft_consensus.cc:504] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.826854 11411 leader_election.cc:302] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.827080 11411 leader_election.cc:288] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.827143 11413 raft_consensus.cc:2781] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.827296 11413 raft_consensus.cc:686] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.827510 11413 consensus_queue.cc:237] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.827884 11411 ts_tablet_manager.cc:1428] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 06:39:50.832234 11113 catalog_manager.cc:5495] T 2c80d691ee644eabb2479a9b72cc4057 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.850248 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:50.862411 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.003s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1162,"drs_written":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4,"lbm_write_time_us":788,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:50.863336 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 600 bytes on disk
I20260809 06:39:50.864060 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.864681 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.276734
I20260809 06:39:50.869227 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53718:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.880856 11297 tablet_service.cc:1467] Processing CreateTablet for tablet ffcda735e16949439fd2af3335a86c40 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.53 [id=e8793a5f7ae84111ac3b7641228e542e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.881428 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcda735e16949439fd2af3335a86c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.887593 11411 tablet_bootstrap.cc:492] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.887727 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.023s sys 0.000s Metrics: {"bytes_written":28791,"cfile_cache_hit":13,"cfile_cache_hit_bytes":20653,"cfile_cache_miss":31,"cfile_cache_miss_bytes":31113,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":583,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1138,"drs_written":1,"lbm_read_time_us":1190,"lbm_reads_lt_1ms":63,"lbm_write_time_us":784,"lbm_writes_lt_1ms":27,"mutex_wait_us":29,"num_input_rowsets":2,"peak_mem_usage":13294,"rows_written":107,"spinlock_wait_cycles":17152}
I20260809 06:39:50.889495 11411 tablet_bootstrap.cc:654] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.890862 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4636 bytes on disk
I20260809 06:39:50.891618 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.892272 11411 tablet_bootstrap.cc:492] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.892446 11411 ts_tablet_manager.cc:1397] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 06:39:50.893056 11411 raft_consensus.cc:348] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.893267 11411 raft_consensus.cc:374] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.893384 11411 raft_consensus.cc:729] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.893682 11411 consensus_queue.cc:260] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.893783 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:50.893879 11411 raft_consensus.cc:388] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.893993 11411 raft_consensus.cc:482] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.894100 11411 raft_consensus.cc:3037] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.896052 11411 raft_consensus.cc:504] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.896330 11411 leader_election.cc:302] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.896595 11411 leader_election.cc:288] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.896675 11413 raft_consensus.cc:2781] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.896857 11413 raft_consensus.cc:686] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.897084 11411 ts_tablet_manager.cc:1428] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:50.897187 11413 consensus_queue.cc:237] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.902181 11113 catalog_manager.cc:5495] T ffcda735e16949439fd2af3335a86c40 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:50.905210 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.001s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":3064,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":910,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:50.906080 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:50.906752 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.907418 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.204939
I20260809 06:39:50.932895 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.021s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":20998,"cfile_cache_miss":28,"cfile_cache_miss_bytes":30938,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":476,"dirs.run_cpu_time_us":407,"dirs.run_wall_time_us":2005,"drs_written":1,"lbm_read_time_us":1123,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1120,"lbm_writes_lt_1ms":31,"mutex_wait_us":85,"num_input_rowsets":2,"peak_mem_usage":14352,"reinsert_count":0,"rows_written":109,"spinlock_wait_cycles":1920,"update_count":1}
I20260809 06:39:50.933931 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4704 bytes on disk
I20260809 06:39:50.934779 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":148,"lbm_reads_lt_1ms":4}
I20260809 06:39:50.945147 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53720:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:50.957165 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 613469fae3db49f083237a937a9031c4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.54 [id=03daa464f120444bb6dbac49f11c952e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:50.957727 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613469fae3db49f083237a937a9031c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:50.962868 11411 tablet_bootstrap.cc:492] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:50.964516 11411 tablet_bootstrap.cc:654] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:50.966708 11411 tablet_bootstrap.cc:492] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:50.966861 11411 ts_tablet_manager.cc:1397] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:50.967680 11411 raft_consensus.cc:348] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.967861 11411 raft_consensus.cc:374] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:50.967993 11411 raft_consensus.cc:729] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:50.968277 11411 consensus_queue.cc:260] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.968489 11411 raft_consensus.cc:388] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:50.968587 11411 raft_consensus.cc:482] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:50.968721 11411 raft_consensus.cc:3037] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:50.970530 11411 raft_consensus.cc:504] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.970788 11411 leader_election.cc:302] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:50.970999 11411 leader_election.cc:288] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:50.971115 11413 raft_consensus.cc:2781] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:50.971354 11413 raft_consensus.cc:686] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:50.971563 11411 ts_tablet_manager.cc:1428] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260809 06:39:50.971597 11413 consensus_queue.cc:237] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:50.976428 11113 catalog_manager.cc:5495] T 613469fae3db49f083237a937a9031c4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.012003 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53732:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.023492 11297 tablet_service.cc:1467] Processing CreateTablet for tablet cacc739cac9846d293dc7e0e3a6551c1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.55 [id=43ee406be26b4e28b715dae7f252b27f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.024027 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cacc739cac9846d293dc7e0e3a6551c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.029450 11411 tablet_bootstrap.cc:492] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.031068 11411 tablet_bootstrap.cc:654] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.033293 11411 tablet_bootstrap.cc:492] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.033514 11411 ts_tablet_manager.cc:1397] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260809 06:39:51.034159 11411 raft_consensus.cc:348] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.034380 11411 raft_consensus.cc:374] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.034474 11411 raft_consensus.cc:729] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.034775 11411 consensus_queue.cc:260] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.034958 11411 raft_consensus.cc:388] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.035079 11411 raft_consensus.cc:482] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.035192 11411 raft_consensus.cc:3037] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.037034 11411 raft_consensus.cc:504] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.037281 11411 leader_election.cc:302] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.037541 11411 leader_election.cc:288] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.037626 11413 raft_consensus.cc:2781] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.037786 11413 raft_consensus.cc:686] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.038023 11413 consensus_queue.cc:237] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.038759 11411 ts_tablet_manager.cc:1428] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260809 06:39:51.043504 11112 catalog_manager.cc:5495] T cacc739cac9846d293dc7e0e3a6551c1 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.057780 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:51.068744 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":232,"dirs.run_wall_time_us":1347,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":859,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:51.069736 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 630 bytes on disk
I20260809 06:39:51.070399 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.070971 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.276012
I20260809 06:39:51.084275 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53736:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.101814 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 089b83c1043c445fbc9bc7cf202b9ddc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.56 [id=d846c7ce9c9e4e39bdbf5abfe0439bfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.102347 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089b83c1043c445fbc9bc7cf202b9ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.105070 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.016s sys 0.009s Metrics: {"bytes_written":30108,"cfile_cache_hit":13,"cfile_cache_hit_bytes":21762,"cfile_cache_miss":31,"cfile_cache_miss_bytes":32704,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":5486,"dirs.run_cpu_time_us":783,"dirs.run_wall_time_us":4765,"drs_written":1,"lbm_read_time_us":1120,"lbm_reads_lt_1ms":63,"lbm_write_time_us":826,"lbm_writes_lt_1ms":27,"mutex_wait_us":4575,"num_input_rowsets":2,"peak_mem_usage":15287,"rows_written":113}
I20260809 06:39:51.106113 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260809 06:39:51.107862 11411 tablet_bootstrap.cc:492] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.109651 11411 tablet_bootstrap.cc:654] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.112121 11411 tablet_bootstrap.cc:492] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.112270 11411 ts_tablet_manager.cc:1397] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:51.113162 11411 raft_consensus.cc:348] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.113412 11411 raft_consensus.cc:374] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.113516 11411 raft_consensus.cc:729] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.113795 11411 consensus_queue.cc:260] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.114089 11411 raft_consensus.cc:388] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.114207 11411 raft_consensus.cc:482] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.114322 11411 raft_consensus.cc:3037] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.116842 11411 raft_consensus.cc:504] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.117152 11411 leader_election.cc:302] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.117420 11411 leader_election.cc:288] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.117503 11413 raft_consensus.cc:2781] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.117682 11413 raft_consensus.cc:686] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.117923 11413 consensus_queue.cc:237] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.117761 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.011s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":216,"dirs.run_wall_time_us":1235,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":944,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":11520}
I20260809 06:39:51.119117 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.171132
I20260809 06:39:51.117925 11411 ts_tablet_manager.cc:1428] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260809 06:39:51.136247 11112 catalog_manager.cc:5495] T 089b83c1043c445fbc9bc7cf202b9ddc P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.149886 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.025s sys 0.003s Metrics: {"bytes_written":30493,"cfile_cache_hit":12,"cfile_cache_hit_bytes":22107,"cfile_cache_miss":28,"cfile_cache_miss_bytes":32587,"cfile_init":9,"delta_iterators_relevant":4,"dirs.queue_time_us":1050,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":1478,"drs_written":1,"lbm_read_time_us":1580,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1807,"lbm_writes_lt_1ms":27,"mutex_wait_us":30,"num_input_rowsets":2,"peak_mem_usage":15228,"rows_written":115,"spinlock_wait_cycles":68224}
I20260809 06:39:51.151060 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260809 06:39:51.155197 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.003s sys 0.000s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":176,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:51.156447 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4973 bytes on disk
I20260809 06:39:51.157055 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.174564 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53752:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.185966 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:51.186093 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 037c7478e04d4e49a6a6a9a6b53040a0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.57 [id=4af4966614574919949712e2dedec12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.186650 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 037c7478e04d4e49a6a6a9a6b53040a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.191812 11411 tablet_bootstrap.cc:492] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.193545 11411 tablet_bootstrap.cc:654] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.195873 11411 tablet_bootstrap.cc:492] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.196064 11411 ts_tablet_manager.cc:1397] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:51.196632 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":6453,"cfile_init":1,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":252,"dirs.run_wall_time_us":1385,"drs_written":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4,"lbm_write_time_us":797,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:51.196738 11411 raft_consensus.cc:348] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.196995 11411 raft_consensus.cc:374] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.197083 11411 raft_consensus.cc:729] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.197407 11411 consensus_queue.cc:260] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.197633 11411 raft_consensus.cc:388] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.197659 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 614 bytes on disk
I20260809 06:39:51.197760 11411 raft_consensus.cc:482] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.197876 11411 raft_consensus.cc:3037] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.198258 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.199025 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.271029
I20260809 06:39:51.200156 11411 raft_consensus.cc:504] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.200397 11411 leader_election.cc:302] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.200632 11411 leader_election.cc:288] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.200695 11413 raft_consensus.cc:2781] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.200860 11413 raft_consensus.cc:686] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.201130 11413 consensus_queue.cc:237] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.202483 11411 ts_tablet_manager.cc:1428] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 06:39:51.206019 11113 catalog_manager.cc:5495] T 037c7478e04d4e49a6a6a9a6b53040a0 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.229147 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.017s sys 0.008s Metrics: {"bytes_written":385,"cfile_cache_hit":13,"cfile_cache_hit_bytes":22461,"cfile_cache_miss":31,"cfile_cache_miss_bytes":33689,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1495,"dirs.run_cpu_time_us":708,"dirs.run_wall_time_us":3402,"drs_written":1,"lbm_read_time_us":3702,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":62,"lbm_write_time_us":1038,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":14565,"reinsert_count":0,"rows_written":117,"update_count":2}
I20260809 06:39:51.230158 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5057 bytes on disk
I20260809 06:39:51.230785 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.253660 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53768:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.265268 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3350dcf0b1ae4ba1892dc7a5d674ee2a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.58 [id=b055838f54f14acdbb819a4f10c7c4d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.265800 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3350dcf0b1ae4ba1892dc7a5d674ee2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.270210 11411 tablet_bootstrap.cc:492] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.271448 11411 tablet_bootstrap.cc:654] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.273603 11411 tablet_bootstrap.cc:492] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.273746 11411 ts_tablet_manager.cc:1397] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:51.274333 11411 raft_consensus.cc:348] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.274499 11411 raft_consensus.cc:374] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.274612 11411 raft_consensus.cc:729] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.274816 11411 consensus_queue.cc:260] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.274992 11411 raft_consensus.cc:388] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.275074 11411 raft_consensus.cc:482] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.275223 11411 raft_consensus.cc:3037] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.277127 11411 raft_consensus.cc:504] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.277443 11411 leader_election.cc:302] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.277644 11411 leader_election.cc:288] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.277808 11413 raft_consensus.cc:2781] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.278018 11413 raft_consensus.cc:686] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.278147 11411 ts_tablet_manager.cc:1428] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:51.278236 11413 consensus_queue.cc:237] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.282342 11113 catalog_manager.cc:5495] T 3350dcf0b1ae4ba1892dc7a5d674ee2a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.311658 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53776:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.323032 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a4d484dc7c7c41249e2023fa65bdbc31 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.59 [id=d438f26cd4eb499cbdc0806237365137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.323539 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d484dc7c7c41249e2023fa65bdbc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.328045 11411 tablet_bootstrap.cc:492] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.329583 11411 tablet_bootstrap.cc:654] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.331629 11411 tablet_bootstrap.cc:492] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.331794 11411 ts_tablet_manager.cc:1397] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:51.332458 11411 raft_consensus.cc:348] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.332674 11411 raft_consensus.cc:374] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.332756 11411 raft_consensus.cc:729] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.332969 11411 consensus_queue.cc:260] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.333199 11411 raft_consensus.cc:388] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.333334 11411 raft_consensus.cc:482] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.333500 11411 raft_consensus.cc:3037] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.335291 11411 raft_consensus.cc:504] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.335520 11411 leader_election.cc:302] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.335721 11411 leader_election.cc:288] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.335837 11413 raft_consensus.cc:2781] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.336040 11413 raft_consensus.cc:686] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.336189 11411 ts_tablet_manager.cc:1428] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:51.336328 11413 consensus_queue.cc:237] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.340992 11112 catalog_manager.cc:5495] T a4d484dc7c7c41249e2023fa65bdbc31 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.374243 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53778:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.385179 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 4c02122a94f544f0911fbbe13c094065 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.60 [id=d0d5c196c2334fc1bd5346fa834bef1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.385737 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c02122a94f544f0911fbbe13c094065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.390689 11411 tablet_bootstrap.cc:492] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.392457 11411 tablet_bootstrap.cc:654] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.394655 11411 tablet_bootstrap.cc:492] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.394778 11411 ts_tablet_manager.cc:1397] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.395457 11411 raft_consensus.cc:348] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.395666 11411 raft_consensus.cc:374] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.395758 11411 raft_consensus.cc:729] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.395949 11411 consensus_queue.cc:260] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.396135 11411 raft_consensus.cc:388] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.396211 11411 raft_consensus.cc:482] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.396302 11411 raft_consensus.cc:3037] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.398976 11411 raft_consensus.cc:504] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.399240 11411 leader_election.cc:302] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.399426 11411 leader_election.cc:288] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.399531 11413 raft_consensus.cc:2781] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.399720 11413 raft_consensus.cc:686] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.399868 11411 ts_tablet_manager.cc:1428] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:51.399933 11413 consensus_queue.cc:237] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.405951 11112 catalog_manager.cc:5495] T 4c02122a94f544f0911fbbe13c094065 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.408107 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:51.428294 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.007s sys 0.003s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":215,"dirs.run_wall_time_us":3392,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1116,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:51.429210 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 604 bytes on disk
I20260809 06:39:51.429991 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.430644 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.274588
I20260809 06:39:51.454391 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53792:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.464351 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.020s sys 0.008s Metrics: {"bytes_written":32275,"cfile_cache_hit":14,"cfile_cache_hit_bytes":23594,"cfile_cache_miss":34,"cfile_cache_miss_bytes":35730,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":1243,"dirs.run_cpu_time_us":1185,"dirs.run_wall_time_us":6629,"drs_written":1,"lbm_read_time_us":1822,"lbm_reads_lt_1ms":70,"lbm_write_time_us":3254,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"mutex_wait_us":46,"num_input_rowsets":2,"peak_mem_usage":15530,"rows_written":123,"spinlock_wait_cycles":1280,"thread_start_us":1578,"threads_started":1}
I20260809 06:39:51.465750 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5330 bytes on disk
I20260809 06:39:51.466600 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.467691 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:51.468675 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 53cc15d7bac64926a16a883cebb41373 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.61 [id=67221a98575449c4bf7f5ab8da7f53f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.469126 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cc15d7bac64926a16a883cebb41373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.475368 11411 tablet_bootstrap.cc:492] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.477087 11411 tablet_bootstrap.cc:654] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.478834 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":161,"dirs.run_wall_time_us":1041,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":727,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:51.479635 11411 tablet_bootstrap.cc:492] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.479806 11411 ts_tablet_manager.cc:1397] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.480425 11411 raft_consensus.cc:348] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.480634 11411 raft_consensus.cc:374] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.480717 11411 raft_consensus.cc:729] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.480997 11411 consensus_queue.cc:260] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.481185 11411 raft_consensus.cc:388] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.481302 11411 raft_consensus.cc:482] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.481438 11411 raft_consensus.cc:3037] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.481873 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:51.481923 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.200248
I20260809 06:39:51.483894 11411 raft_consensus.cc:504] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.484157 11411 leader_election.cc:302] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.484382 11411 leader_election.cc:288] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.484462 11413 raft_consensus.cc:2781] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.484637 11413 raft_consensus.cc:686] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.484906 11411 ts_tablet_manager.cc:1428] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:51.484901 11413 consensus_queue.cc:237] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.493629 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.004s Metrics: {"bytes_written":6454,"cfile_init":1,"dirs.queue_time_us":749,"dirs.run_cpu_time_us":267,"dirs.run_wall_time_us":1128,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":883,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:51.495261 11112 catalog_manager.cc:5495] T 53cc15d7bac64926a16a883cebb41373 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.496805 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 584 bytes on disk
I20260809 06:39:51.497951 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.499226 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.262422
I20260809 06:39:51.509302 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.004s Metrics: {"bytes_written":32677,"cfile_cache_hit":12,"cfile_cache_hit_bytes":23942,"cfile_cache_miss":28,"cfile_cache_miss_bytes":35278,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":664,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":1006,"drs_written":1,"lbm_read_time_us":1564,"lbm_reads_lt_1ms":60,"lbm_write_time_us":995,"lbm_writes_lt_1ms":27,"mutex_wait_us":135,"num_input_rowsets":2,"peak_mem_usage":16701,"rows_written":125,"spinlock_wait_cycles":4992}
I20260809 06:39:51.510238 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5399 bytes on disk
I20260809 06:39:51.510804 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.511353 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000002
I20260809 06:39:51.516806 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.001s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":139,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:51.543275 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53808:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.544452 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.021s sys 0.004s Metrics: {"bytes_written":32714,"cfile_cache_hit":13,"cfile_cache_hit_bytes":23933,"cfile_cache_miss":31,"cfile_cache_miss_bytes":35911,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":746,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1163,"drs_written":1,"lbm_read_time_us":775,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1019,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":15986,"rows_written":125}
I20260809 06:39:51.545379 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5415 bytes on disk
I20260809 06:39:51.546031 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.555377 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 87f574f4313a4a5f846ebbbd720dd464 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.62 [id=44e33f2bdecf42749d96d0e4b5b6590b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.555941 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f574f4313a4a5f846ebbbd720dd464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.560642 11411 tablet_bootstrap.cc:492] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.562170 11411 tablet_bootstrap.cc:654] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.564312 11411 tablet_bootstrap.cc:492] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.564431 11411 ts_tablet_manager.cc:1397] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.565011 11411 raft_consensus.cc:348] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.565161 11411 raft_consensus.cc:374] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.565219 11411 raft_consensus.cc:729] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.565474 11411 consensus_queue.cc:260] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.565706 11411 raft_consensus.cc:388] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.565809 11411 raft_consensus.cc:482] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.565948 11411 raft_consensus.cc:3037] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.567771 11411 raft_consensus.cc:504] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.568022 11411 leader_election.cc:302] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.568215 11411 leader_election.cc:288] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.568334 11413 raft_consensus.cc:2781] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.568568 11413 raft_consensus.cc:686] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.568728 11411 ts_tablet_manager.cc:1428] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:51.568845 11413 consensus_queue.cc:237] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.573647 11113 catalog_manager.cc:5495] T 87f574f4313a4a5f846ebbbd720dd464 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.607483 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53814:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.618330 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c159c9967abe4264877287c477da6d4a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.63 [id=456476187316436685b7a3bbed35895b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.618722 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c159c9967abe4264877287c477da6d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.623616 11411 tablet_bootstrap.cc:492] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.624823 11411 tablet_bootstrap.cc:654] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.627560 11411 tablet_bootstrap.cc:492] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.627740 11411 ts_tablet_manager.cc:1397] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:51.628361 11411 raft_consensus.cc:348] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.628587 11411 raft_consensus.cc:374] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.628682 11411 raft_consensus.cc:729] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.628950 11411 consensus_queue.cc:260] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.629127 11411 raft_consensus.cc:388] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.629256 11411 raft_consensus.cc:482] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.629395 11411 raft_consensus.cc:3037] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.631527 11411 raft_consensus.cc:504] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.631764 11411 leader_election.cc:302] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.632014 11411 leader_election.cc:288] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.632085 11413 raft_consensus.cc:2781] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.632246 11413 raft_consensus.cc:686] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.632525 11411 ts_tablet_manager.cc:1428] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:51.632522 11413 consensus_queue.cc:237] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.636989 11113 catalog_manager.cc:5495] T c159c9967abe4264877287c477da6d4a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.672240 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53828:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.683388 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 56c6b0eaa2534edf9732a331bc1881c1 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.64 [id=47a03d013f6f41e3b81ef37578ad939a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.683895 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56c6b0eaa2534edf9732a331bc1881c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.688756 11411 tablet_bootstrap.cc:492] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.690419 11411 tablet_bootstrap.cc:654] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.692775 11411 tablet_bootstrap.cc:492] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.692939 11411 ts_tablet_manager.cc:1397] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.693830 11411 raft_consensus.cc:348] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.694033 11411 raft_consensus.cc:374] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.694142 11411 raft_consensus.cc:729] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.694356 11411 consensus_queue.cc:260] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.694612 11411 raft_consensus.cc:388] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.694689 11411 raft_consensus.cc:482] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.694806 11411 raft_consensus.cc:3037] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.697183 11411 raft_consensus.cc:504] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.697451 11411 leader_election.cc:302] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.697639 11411 leader_election.cc:288] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.697724 11413 raft_consensus.cc:2781] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.697949 11413 raft_consensus.cc:686] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.698158 11411 ts_tablet_manager.cc:1428] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:51.698184 11413 consensus_queue.cc:237] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.702731 11113 catalog_manager.cc:5495] T 56c6b0eaa2534edf9732a331bc1881c1 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.732157 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53844:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.743237 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c0eb46eb8b9b473dbc72eab4bd266d23 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.65 [id=affffcc181574c95bed58739d08f0bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.743741 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0eb46eb8b9b473dbc72eab4bd266d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.748484 11411 tablet_bootstrap.cc:492] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.749922 11411 tablet_bootstrap.cc:654] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.752218 11411 tablet_bootstrap.cc:492] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.752342 11411 ts_tablet_manager.cc:1397] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.752887 11411 raft_consensus.cc:348] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.753063 11411 raft_consensus.cc:374] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.753119 11411 raft_consensus.cc:729] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.753290 11411 consensus_queue.cc:260] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.753510 11411 raft_consensus.cc:388] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.753609 11411 raft_consensus.cc:482] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.753702 11411 raft_consensus.cc:3037] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.756031 11411 raft_consensus.cc:504] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.756362 11411 leader_election.cc:302] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.756572 11411 leader_election.cc:288] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.756667 11413 raft_consensus.cc:2781] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.756860 11413 raft_consensus.cc:686] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.757092 11411 ts_tablet_manager.cc:1428] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:51.757236 11413 consensus_queue.cc:237] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.761639 11113 catalog_manager.cc:5495] T c0eb46eb8b9b473dbc72eab4bd266d23 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.767587 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:51.779107 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":6489,"cfile_init":1,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1332,"drs_written":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4,"lbm_write_time_us":709,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:51.779942 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.262632
I20260809 06:39:51.796998 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260809 06:39:51.799324 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53854:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.809659 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.003s Metrics: {"bytes_written":6489,"cfile_init":1,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":1077,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":630,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:51.811133 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.020s sys 0.005s Metrics: {"bytes_written":34476,"cfile_cache_hit":13,"cfile_cache_hit_bytes":25442,"cfile_cache_miss":31,"cfile_cache_miss_bytes":38024,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":804,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1117,"drs_written":1,"lbm_read_time_us":953,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1022,"lbm_writes_lt_1ms":28,"mutex_wait_us":247,"num_input_rowsets":2,"peak_mem_usage":18030,"rows_written":133,"spinlock_wait_cycles":1408}
I20260809 06:39:51.811452 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.263276
I20260809 06:39:51.811405 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1105dc0c4f864bd0b8a6b0e2a861d89b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.66 [id=9f65c5eaf3f24d8d80ac696311cfac23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.811832 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1105dc0c4f864bd0b8a6b0e2a861d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.811894 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:51.816308 11411 tablet_bootstrap.cc:492] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.817811 11411 tablet_bootstrap.cc:654] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.820827 11411 tablet_bootstrap.cc:492] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.820993 11411 ts_tablet_manager.cc:1397] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:51.821689 11411 raft_consensus.cc:348] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.821871 11411 raft_consensus.cc:374] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.822000 11411 raft_consensus.cc:729] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.822234 11411 consensus_queue.cc:260] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.822443 11411 raft_consensus.cc:388] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.822540 11411 raft_consensus.cc:482] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.822676 11411 raft_consensus.cc:3037] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.823583 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":95,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1162,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":952,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:51.824371 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.194124
I20260809 06:39:51.824422 11411 raft_consensus.cc:504] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.824735 11411 leader_election.cc:302] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.824923 11411 leader_election.cc:288] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.824983 11413 raft_consensus.cc:2781] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.825425 11411 ts_tablet_manager.cc:1428] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:51.825569 11413 raft_consensus.cc:686] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.825824 11413 consensus_queue.cc:237] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.833161 11113 catalog_manager.cc:5495] T 1105dc0c4f864bd0b8a6b0e2a861d89b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.847313 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.020s sys 0.008s Metrics: {"bytes_written":34476,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25445,"cfile_cache_miss":28,"cfile_cache_miss_bytes":37663,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":873,"dirs.run_cpu_time_us":846,"dirs.run_wall_time_us":5904,"drs_written":1,"lbm_read_time_us":1352,"lbm_reads_lt_1ms":60,"lbm_write_time_us":970,"lbm_writes_lt_1ms":28,"mutex_wait_us":223,"num_input_rowsets":2,"peak_mem_usage":17977,"rows_written":133}
I20260809 06:39:51.848196 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:51.855963 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.027s sys 0.003s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25787,"cfile_cache_miss":28,"cfile_cache_miss_bytes":37963,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":670,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1051,"drs_written":1,"lbm_read_time_us":1377,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1314,"lbm_writes_lt_1ms":32,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":17971,"reinsert_count":0,"rows_written":135,"spinlock_wait_cycles":3456,"update_count":1}
I20260809 06:39:51.856946 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5829 bytes on disk
I20260809 06:39:51.857611 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":123,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.860556 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":5193,"cfile_init":1,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":975,"drs_written":1,"lbm_read_time_us":117,"lbm_reads_lt_1ms":4,"lbm_write_time_us":859,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:51.861456 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6114 bytes on disk
I20260809 06:39:51.863267 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":191,"lbm_reads_lt_1ms":8}
I20260809 06:39:51.863972 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.195271
I20260809 06:39:51.874033 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53870:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.886267 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0efe2798574eccb88b7b526867474c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.67 [id=a041d05a83e04222858d7b53070d352d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.886754 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0efe2798574eccb88b7b526867474c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.890694 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.021s sys 0.004s Metrics: {"bytes_written":34909,"cfile_cache_hit":12,"cfile_cache_hit_bytes":25825,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38241,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":841,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":1849,"drs_written":1,"lbm_read_time_us":1178,"lbm_reads_lt_1ms":56,"lbm_write_time_us":925,"lbm_writes_lt_1ms":28,"mutex_wait_us":102,"num_input_rowsets":2,"peak_mem_usage":17971,"rows_written":135,"spinlock_wait_cycles":16512}
I20260809 06:39:51.891623 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:51.892139 11411 tablet_bootstrap.cc:492] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.893489 11411 tablet_bootstrap.cc:654] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.895850 11411 tablet_bootstrap.cc:492] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.896018 11411 ts_tablet_manager.cc:1397] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 06:39:51.896667 11411 raft_consensus.cc:348] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.896878 11411 raft_consensus.cc:374] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.896965 11411 raft_consensus.cc:729] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.897197 11411 consensus_queue.cc:260] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.897393 11411 raft_consensus.cc:388] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.897519 11411 raft_consensus.cc:482] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.897636 11411 raft_consensus.cc:3037] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.899493 11411 raft_consensus.cc:504] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.899741 11411 leader_election.cc:302] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.899998 11411 leader_election.cc:288] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.900058 11413 raft_consensus.cc:2781] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.900249 11413 raft_consensus.cc:686] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.900552 11411 ts_tablet_manager.cc:1428] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:51.900493 11413 consensus_queue.cc:237] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.901957 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.010s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":126,"dirs.run_wall_time_us":1007,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":775,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":896}
I20260809 06:39:51.902796 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.170148
I20260809 06:39:51.905839 11113 catalog_manager.cc:5495] T 2b0efe2798574eccb88b7b526867474c P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:51.929000 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.022s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":26150,"cfile_cache_miss":28,"cfile_cache_miss_bytes":38498,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":806,"dirs.run_cpu_time_us":230,"dirs.run_wall_time_us":1920,"drs_written":1,"lbm_read_time_us":1217,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1215,"lbm_writes_lt_1ms":32,"mutex_wait_us":22,"num_input_rowsets":2,"peak_mem_usage":18225,"reinsert_count":0,"rows_written":137,"update_count":1}
I20260809 06:39:51.929867 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5910 bytes on disk
I20260809 06:39:51.930397 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4}
I20260809 06:39:51.943243 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53886:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:51.954043 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 450459d4fdd24397baa31d90d937c0a2 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.68 [id=9c32db27254b4941b2474cb7fc9f1c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:51.954563 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450459d4fdd24397baa31d90d937c0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:51.959308 11411 tablet_bootstrap.cc:492] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:51.960812 11411 tablet_bootstrap.cc:654] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:51.962958 11411 tablet_bootstrap.cc:492] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:51.963088 11411 ts_tablet_manager.cc:1397] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:51.963847 11411 raft_consensus.cc:348] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.964006 11411 raft_consensus.cc:374] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:51.964130 11411 raft_consensus.cc:729] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:51.964356 11411 consensus_queue.cc:260] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.964567 11411 raft_consensus.cc:388] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:51.964658 11411 raft_consensus.cc:482] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:51.964785 11411 raft_consensus.cc:3037] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:51.966679 11411 raft_consensus.cc:504] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.966881 11411 leader_election.cc:302] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:51.967051 11411 leader_election.cc:288] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:51.967123 11413 raft_consensus.cc:2781] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:51.967276 11413 raft_consensus.cc:686] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:51.967474 11413 consensus_queue.cc:237] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:51.967803 11411 ts_tablet_manager.cc:1428] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:51.973485 11113 catalog_manager.cc:5495] T 450459d4fdd24397baa31d90d937c0a2 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.004835 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53900:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.015829 11297 tablet_service.cc:1467] Processing CreateTablet for tablet ccc072353d2942a9b84937ce5bd14764 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.69 [id=df9f0bc3ac8442c39dcf1aa5be250357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.016335 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc072353d2942a9b84937ce5bd14764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.020915 11411 tablet_bootstrap.cc:492] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.022356 11411 tablet_bootstrap.cc:654] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.024677 11411 tablet_bootstrap.cc:492] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.024806 11411 ts_tablet_manager.cc:1397] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.025439 11411 raft_consensus.cc:348] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.025607 11411 raft_consensus.cc:374] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.025660 11411 raft_consensus.cc:729] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.025822 11411 consensus_queue.cc:260] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.025945 11411 raft_consensus.cc:388] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.026031 11411 raft_consensus.cc:482] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.026099 11411 raft_consensus.cc:3037] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.027895 11411 raft_consensus.cc:504] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.028147 11411 leader_election.cc:302] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.028355 11411 leader_election.cc:288] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.028469 11413 raft_consensus.cc:2781] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.028672 11413 raft_consensus.cc:686] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.028821 11411 ts_tablet_manager.cc:1428] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260809 06:39:52.028905 11413 consensus_queue.cc:237] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.034574 11112 catalog_manager.cc:5495] T ccc072353d2942a9b84937ce5bd14764 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.070425 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53906:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.081341 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c3b34b2463244c2299d44c8c524e8524 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.70 [id=48832e361e3f478aa15dc812b23ea1a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.081871 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3b34b2463244c2299d44c8c524e8524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.086486 11411 tablet_bootstrap.cc:492] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.088127 11411 tablet_bootstrap.cc:654] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.090469 11411 tablet_bootstrap.cc:492] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.090585 11411 ts_tablet_manager.cc:1397] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:52.091168 11411 raft_consensus.cc:348] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.091306 11411 raft_consensus.cc:374] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.091398 11411 raft_consensus.cc:729] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.091640 11411 consensus_queue.cc:260] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.091883 11411 raft_consensus.cc:388] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.091980 11411 raft_consensus.cc:482] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.092098 11411 raft_consensus.cc:3037] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.093994 11411 raft_consensus.cc:504] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.094274 11411 leader_election.cc:302] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.094471 11411 leader_election.cc:288] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.094599 11413 raft_consensus.cc:2781] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.094825 11413 raft_consensus.cc:686] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.095006 11411 ts_tablet_manager.cc:1428] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:52.095103 11413 consensus_queue.cc:237] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.099339 11112 catalog_manager.cc:5495] T c3b34b2463244c2299d44c8c524e8524 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.108485 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:52.120971 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.003s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1205,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":769,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8,"spinlock_wait_cycles":15616}
I20260809 06:39:52.121964 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259054
I20260809 06:39:52.142513 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53908:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.148243 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.018s sys 0.008s Metrics: {"bytes_written":36652,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27277,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40727,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":985,"dirs.run_cpu_time_us":194,"dirs.run_wall_time_us":1261,"drs_written":1,"lbm_read_time_us":938,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1025,"lbm_writes_lt_1ms":28,"mutex_wait_us":22,"num_input_rowsets":2,"peak_mem_usage":19300,"rows_written":143,"spinlock_wait_cycles":20736}
I20260809 06:39:52.149475 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000008
I20260809 06:39:52.155174 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a7e0ff043d8e45c59e209979f240dba6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.71 [id=4d9f2649caa340f695d619721acdbbf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.155692 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7e0ff043d8e45c59e209979f240dba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.160192 11411 tablet_bootstrap.cc:492] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.161926 11411 tablet_bootstrap.cc:654] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.161693 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1083,"drs_written":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1216,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:52.162660 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.538588
I20260809 06:39:52.164867 11411 tablet_bootstrap.cc:492] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.165027 11411 ts_tablet_manager.cc:1397] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:52.165759 11411 raft_consensus.cc:348] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.165989 11411 raft_consensus.cc:374] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.166088 11411 raft_consensus.cc:729] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.166401 11411 consensus_queue.cc:260] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.166669 11411 raft_consensus.cc:388] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.166811 11411 raft_consensus.cc:482] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.166927 11411 raft_consensus.cc:3037] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.168613 11411 raft_consensus.cc:504] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.168855 11411 leader_election.cc:302] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.169085 11411 leader_election.cc:288] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.169176 11413 raft_consensus.cc:2781] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.169430 11413 raft_consensus.cc:686] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.169637 11411 ts_tablet_manager.cc:1428] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:52.169683 11413 consensus_queue.cc:237] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.170888 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":458,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":122,"lbm_read_time_us":363,"lbm_reads_lt_1ms":25,"lbm_write_time_us":273,"lbm_writes_lt_1ms":9,"mutex_wait_us":24,"reinsert_count":0,"update_count":1}
I20260809 06:39:52.171535 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6338 bytes on disk
I20260809 06:39:52.172281 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":160,"lbm_reads_lt_1ms":8}
I20260809 06:39:52.172912 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.225185
I20260809 06:39:52.178086 11112 catalog_manager.cc:5495] T a7e0ff043d8e45c59e209979f240dba6 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.181418 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:52.196977 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.011s sys 0.001s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1160,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1135,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:52.198686 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.266103
I20260809 06:39:52.204934 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.028s sys 0.001s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":27655,"cfile_cache_miss":31,"cfile_cache_miss_bytes":40703,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1311,"dirs.run_cpu_time_us":414,"dirs.run_wall_time_us":2436,"drs_written":1,"lbm_read_time_us":1176,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1123,"lbm_writes_lt_1ms":32,"mutex_wait_us":46,"num_input_rowsets":2,"peak_mem_usage":19352,"reinsert_count":0,"rows_written":145,"thread_start_us":273,"threads_started":1,"update_count":1}
I20260809 06:39:52.205947 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6241 bytes on disk
I20260809 06:39:52.206521 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.222589 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53916:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.225538 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.004s Metrics: {"bytes_written":37090,"cfile_cache_hit":14,"cfile_cache_hit_bytes":27647,"cfile_cache_miss":34,"cfile_cache_miss_bytes":41631,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":2676,"dirs.run_cpu_time_us":454,"dirs.run_wall_time_us":2471,"drs_written":1,"lbm_read_time_us":1092,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1072,"lbm_writes_lt_1ms":28,"mutex_wait_us":42,"num_input_rowsets":2,"peak_mem_usage":19607,"rows_written":145,"spinlock_wait_cycles":2176,"thread_start_us":476,"threads_started":1}
I20260809 06:39:52.226403 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6252 bytes on disk
I20260809 06:39:52.227178 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.234833 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 62c89cd3794f463f829dc685cb01258f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.72 [id=de01e61e74c04debb94bd2ad932f6320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.235378 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c89cd3794f463f829dc685cb01258f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.239874 11411 tablet_bootstrap.cc:492] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.241466 11411 tablet_bootstrap.cc:654] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.243809 11411 tablet_bootstrap.cc:492] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.243916 11411 ts_tablet_manager.cc:1397] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.244431 11411 raft_consensus.cc:348] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.244601 11411 raft_consensus.cc:374] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.244666 11411 raft_consensus.cc:729] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.244889 11411 consensus_queue.cc:260] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.245046 11411 raft_consensus.cc:388] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.245101 11411 raft_consensus.cc:482] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.245194 11411 raft_consensus.cc:3037] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.247190 11411 raft_consensus.cc:504] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.247396 11411 leader_election.cc:302] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.247568 11411 leader_election.cc:288] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.247641 11413 raft_consensus.cc:2781] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.247823 11413 raft_consensus.cc:686] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.248137 11411 ts_tablet_manager.cc:1428] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:52.248042 11413 consensus_queue.cc:237] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.254566 11112 catalog_manager.cc:5495] T 62c89cd3794f463f829dc685cb01258f P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.287427 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44856:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.298672 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 6935c7ecc00d4f37946008f2bbd0ad9a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.73 [id=89bec41c273f457f8ca1d4e5f0241365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.299165 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935c7ecc00d4f37946008f2bbd0ad9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.303643 11411 tablet_bootstrap.cc:492] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.304963 11411 tablet_bootstrap.cc:654] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.307240 11411 tablet_bootstrap.cc:492] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.307408 11411 ts_tablet_manager.cc:1397] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:52.307996 11411 raft_consensus.cc:348] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.308205 11411 raft_consensus.cc:374] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.308300 11411 raft_consensus.cc:729] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.308560 11411 consensus_queue.cc:260] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.308728 11411 raft_consensus.cc:388] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.308830 11411 raft_consensus.cc:482] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.308926 11411 raft_consensus.cc:3037] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.310853 11411 raft_consensus.cc:504] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.311087 11411 leader_election.cc:302] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.311324 11411 leader_election.cc:288] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.311440 11413 raft_consensus.cc:2781] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.311616 11413 raft_consensus.cc:686] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.311801 11411 ts_tablet_manager.cc:1428] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:52.311852 11413 consensus_queue.cc:237] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.317493 11112 catalog_manager.cc:5495] T 6935c7ecc00d4f37946008f2bbd0ad9a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.351262 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44868:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.362344 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 78ab8a67b3f249fcbe0333c080c946a4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.74 [id=02c6f66a10dd4bfbb75c55f34f673d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.362712 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78ab8a67b3f249fcbe0333c080c946a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.367223 11411 tablet_bootstrap.cc:492] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.368458 11411 tablet_bootstrap.cc:654] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.370805 11411 tablet_bootstrap.cc:492] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.370958 11411 ts_tablet_manager.cc:1397] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:52.371564 11411 raft_consensus.cc:348] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.371745 11411 raft_consensus.cc:374] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.371857 11411 raft_consensus.cc:729] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.372069 11411 consensus_queue.cc:260] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.372257 11411 raft_consensus.cc:388] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.372350 11411 raft_consensus.cc:482] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.372474 11411 raft_consensus.cc:3037] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.374431 11411 raft_consensus.cc:504] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.374730 11411 leader_election.cc:302] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.374919 11411 leader_election.cc:288] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.374981 11413 raft_consensus.cc:2781] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.375185 11413 raft_consensus.cc:686] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.375455 11411 ts_tablet_manager.cc:1428] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.375423 11413 consensus_queue.cc:237] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.380632 11112 catalog_manager.cc:5495] T 78ab8a67b3f249fcbe0333c080c946a4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.428208 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44870:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.440971 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 56c254861ea3476aa71d0f3d00861601 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.75 [id=557a6145c9cb418e8404b2d543dee63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.441538 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56c254861ea3476aa71d0f3d00861601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.446159 11411 tablet_bootstrap.cc:492] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.447906 11411 tablet_bootstrap.cc:654] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.450712 11411 tablet_bootstrap.cc:492] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.450881 11411 ts_tablet_manager.cc:1397] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:52.451658 11411 raft_consensus.cc:348] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.451903 11411 raft_consensus.cc:374] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.451995 11411 raft_consensus.cc:729] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.452234 11411 consensus_queue.cc:260] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.452431 11411 raft_consensus.cc:388] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.452535 11411 raft_consensus.cc:482] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.452636 11411 raft_consensus.cc:3037] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.455628 11411 raft_consensus.cc:504] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.455912 11411 leader_election.cc:302] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.456157 11411 leader_election.cc:288] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.456282 11413 raft_consensus.cc:2781] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.456471 11413 raft_consensus.cc:686] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.456682 11411 ts_tablet_manager.cc:1428] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260809 06:39:52.456781 11413 consensus_queue.cc:237] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.457448 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260809 06:39:52.463689 11112 catalog_manager.cc:5495] T 56c254861ea3476aa71d0f3d00861601 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.471271 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.000s Metrics: {"bytes_written":6454,"cfile_init":1,"dirs.queue_time_us":457,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":3752,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":797,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:52.472095 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 587 bytes on disk
I20260809 06:39:52.474116 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.474809 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.231088
I20260809 06:39:52.478204 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:52.490852 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.002s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":1069,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":858,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:52.491756 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 598 bytes on disk
I20260809 06:39:52.492419 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.493055 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.232563
I20260809 06:39:52.515090 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.026s sys 0.004s Metrics: {"bytes_written":38827,"cfile_cache_hit":13,"cfile_cache_hit_bytes":29085,"cfile_cache_miss":31,"cfile_cache_miss_bytes":43335,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":351,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":1093,"drs_written":1,"lbm_read_time_us":1332,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1116,"lbm_writes_lt_1ms":28,"mutex_wait_us":86,"num_input_rowsets":2,"peak_mem_usage":19548,"rows_written":153}
I20260809 06:39:52.516233 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6582 bytes on disk
I20260809 06:39:52.516830 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.523329 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.022s sys 0.007s Metrics: {"bytes_written":38827,"cfile_cache_hit":12,"cfile_cache_hit_bytes":29125,"cfile_cache_miss":28,"cfile_cache_miss_bytes":43023,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":513,"dirs.run_cpu_time_us":208,"dirs.run_wall_time_us":1062,"drs_written":1,"lbm_read_time_us":1218,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1321,"lbm_writes_lt_1ms":28,"mutex_wait_us":3,"num_input_rowsets":2,"peak_mem_usage":19495,"rows_written":153,"spinlock_wait_cycles":1792}
I20260809 06:39:52.524394 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6582 bytes on disk
I20260809 06:39:52.525575 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.527522 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44872:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.539296 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d78c7dd2a0064b6a89a61def4b2e6990 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.76 [id=c27d633e053c438dbe6b835926282f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.539764 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78c7dd2a0064b6a89a61def4b2e6990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.544247 11411 tablet_bootstrap.cc:492] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.545737 11411 tablet_bootstrap.cc:654] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.548034 11411 tablet_bootstrap.cc:492] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.548154 11411 ts_tablet_manager.cc:1397] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.548744 11411 raft_consensus.cc:348] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.548900 11411 raft_consensus.cc:374] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.548960 11411 raft_consensus.cc:729] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.549201 11411 consensus_queue.cc:260] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.549417 11411 raft_consensus.cc:388] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.549486 11411 raft_consensus.cc:482] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.549573 11411 raft_consensus.cc:3037] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.551491 11411 raft_consensus.cc:504] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.551698 11411 leader_election.cc:302] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.551870 11411 leader_election.cc:288] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.551947 11413 raft_consensus.cc:2781] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.552115 11413 raft_consensus.cc:686] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.552392 11411 ts_tablet_manager.cc:1428] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:52.552337 11413 consensus_queue.cc:237] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.557551 11113 catalog_manager.cc:5495] T d78c7dd2a0064b6a89a61def4b2e6990 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.591252 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44880:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.602023 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 25beb289620c4745aa970b8475e52642 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.77 [id=38d9f09d5120437ebfa1e716ddeeb583]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.602502 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25beb289620c4745aa970b8475e52642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.607290 11411 tablet_bootstrap.cc:492] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.608536 11411 tablet_bootstrap.cc:654] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.611013 11411 tablet_bootstrap.cc:492] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.611186 11411 ts_tablet_manager.cc:1397] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:52.611768 11411 raft_consensus.cc:348] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.611965 11411 raft_consensus.cc:374] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.612051 11411 raft_consensus.cc:729] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.612275 11411 consensus_queue.cc:260] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.612447 11411 raft_consensus.cc:388] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.612565 11411 raft_consensus.cc:482] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.612672 11411 raft_consensus.cc:3037] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.614728 11411 raft_consensus.cc:504] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.614989 11411 leader_election.cc:302] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.615206 11411 leader_election.cc:288] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.615265 11413 raft_consensus.cc:2781] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.615401 11413 raft_consensus.cc:686] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.615636 11413 consensus_queue.cc:237] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.615782 11411 ts_tablet_manager.cc:1428] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.621193 11113 catalog_manager.cc:5495] T 25beb289620c4745aa970b8475e52642 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.653596 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44896:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.665413 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 6f664f16c91947c8899040bd43cd0814 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.78 [id=e0dd370457554c248b780e81371ba11a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.665846 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f664f16c91947c8899040bd43cd0814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.671650 11411 tablet_bootstrap.cc:492] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.673552 11411 tablet_bootstrap.cc:654] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.675736 11411 tablet_bootstrap.cc:492] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.675864 11411 ts_tablet_manager.cc:1397] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:52.676543 11411 raft_consensus.cc:348] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.676708 11411 raft_consensus.cc:374] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.676867 11411 raft_consensus.cc:729] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.677143 11411 consensus_queue.cc:260] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.677413 11411 raft_consensus.cc:388] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.677531 11411 raft_consensus.cc:482] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.677664 11411 raft_consensus.cc:3037] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.679580 11411 raft_consensus.cc:504] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.679889 11411 leader_election.cc:302] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.680195 11411 leader_election.cc:288] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.680279 11413 raft_consensus.cc:2781] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.680496 11413 raft_consensus.cc:686] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.680721 11411 ts_tablet_manager.cc:1428] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:52.680802 11413 consensus_queue.cc:237] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.686635 11112 catalog_manager.cc:5495] T 6f664f16c91947c8899040bd43cd0814 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.724068 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44898:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.735145 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbcf4f1bfda4328b89cb8d0d9f9ea1b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.79 [id=73fd57c8f24747c8869e4bbd526ed1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.735626 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbcf4f1bfda4328b89cb8d0d9f9ea1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.739931 11411 tablet_bootstrap.cc:492] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.741508 11411 tablet_bootstrap.cc:654] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.743984 11411 tablet_bootstrap.cc:492] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.744110 11411 ts_tablet_manager.cc:1397] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:52.744635 11411 raft_consensus.cc:348] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.744791 11411 raft_consensus.cc:374] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.744844 11411 raft_consensus.cc:729] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.745083 11411 consensus_queue.cc:260] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.745219 11411 raft_consensus.cc:388] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.745288 11411 raft_consensus.cc:482] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.745424 11411 raft_consensus.cc:3037] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.747112 11411 raft_consensus.cc:504] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.747339 11411 leader_election.cc:302] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.747527 11411 leader_election.cc:288] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.747588 11413 raft_consensus.cc:2781] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.747776 11413 raft_consensus.cc:686] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.748027 11413 consensus_queue.cc:237] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.748350 11411 ts_tablet_manager.cc:1428] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260809 06:39:52.753190 11112 catalog_manager.cc:5495] T 7bbcf4f1bfda4328b89cb8d0d9f9ea1b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.768378 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260809 06:39:52.776464 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:52.778425 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.007s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":1356,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":674,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:52.779667 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 596 bytes on disk
I20260809 06:39:52.780251 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.780900 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.254992
I20260809 06:39:52.789667 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.003s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":699,"dirs.run_cpu_time_us":145,"dirs.run_wall_time_us":1117,"drs_written":1,"lbm_read_time_us":461,"lbm_reads_lt_1ms":4,"lbm_write_time_us":981,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8,"spinlock_wait_cycles":9984}
I20260809 06:39:52.790598 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.254992
I20260809 06:39:52.793469 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44910:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.807974 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 98873ec97bc34307b51d228ab0bfd7e9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.80 [id=750acc71636245498df2e2f1a6792736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.808504 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98873ec97bc34307b51d228ab0bfd7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.809019 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.023s sys 0.004s Metrics: {"bytes_written":40579,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30597,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45157,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":966,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":2435,"drs_written":1,"lbm_read_time_us":1196,"lbm_reads_lt_1ms":56,"lbm_write_time_us":996,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":21533,"rows_written":161}
I20260809 06:39:52.810653 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:52.814915 11411 tablet_bootstrap.cc:492] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.816478 11411 tablet_bootstrap.cc:654] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.818919 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.027s sys 0.000s Metrics: {"bytes_written":40579,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30597,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45157,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":574,"dirs.run_cpu_time_us":231,"dirs.run_wall_time_us":2370,"drs_written":1,"lbm_read_time_us":1158,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1118,"lbm_writes_lt_1ms":28,"mutex_wait_us":56,"num_input_rowsets":2,"peak_mem_usage":21533,"rows_written":161,"spinlock_wait_cycles":13184}
I20260809 06:39:52.819871 11411 tablet_bootstrap.cc:492] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.820017 11411 ts_tablet_manager.cc:1397] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:52.820730 11411 raft_consensus.cc:348] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.820921 11411 raft_consensus.cc:374] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.821067 11411 raft_consensus.cc:729] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.821376 11411 consensus_queue.cc:260] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.821512 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:52.821319 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":1064,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":876,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":18176}
I20260809 06:39:52.822084 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7148 bytes on disk
I20260809 06:39:52.821630 11411 raft_consensus.cc:388] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.822329 11411 raft_consensus.cc:482] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.822463 11411 raft_consensus.cc:3037] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.822944 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":167,"lbm_reads_lt_1ms":8}
I20260809 06:39:52.823536 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.196647
I20260809 06:39:52.825146 11411 raft_consensus.cc:504] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.825474 11411 leader_election.cc:302] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.825677 11411 leader_election.cc:288] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.825735 11413 raft_consensus.cc:2781] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.825924 11413 raft_consensus.cc:686] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.826136 11413 consensus_queue.cc:237] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.826479 11411 ts_tablet_manager.cc:1428] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 06:39:52.833909 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.004s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":39,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":958,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":966,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:52.834687 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7148 bytes on disk
I20260809 06:39:52.835381 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":157,"lbm_reads_lt_1ms":8}
I20260809 06:39:52.836007 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.196647
I20260809 06:39:52.839938 11112 catalog_manager.cc:5495] T 98873ec97bc34307b51d228ab0bfd7e9 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.855403 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.022s sys 0.007s Metrics: {"bytes_written":40962,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30939,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45463,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":773,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":1345,"drs_written":1,"lbm_read_time_us":1176,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1234,"lbm_writes_lt_1ms":28,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":21621,"rows_written":163}
I20260809 06:39:52.856467 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6978 bytes on disk
I20260809 06:39:52.857172 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.857815 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000003
I20260809 06:39:52.862906 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.002s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":141,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:52.869451 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.024s sys 0.005s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":30939,"cfile_cache_miss":28,"cfile_cache_miss_bytes":45463,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":945,"dirs.run_cpu_time_us":553,"dirs.run_wall_time_us":2765,"drs_written":1,"lbm_read_time_us":1161,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1236,"lbm_writes_lt_1ms":32,"mutex_wait_us":387,"num_input_rowsets":2,"peak_mem_usage":21621,"reinsert_count":0,"rows_written":163,"update_count":1}
I20260809 06:39:52.871783 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 6978 bytes on disk
I20260809 06:39:52.873009 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260809 06:39:52.877842 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44912:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.888873 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7d3bf0097349fa8255459ed39b4596 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.81 [id=3ca014a2da7a4e7b8c3aecf050586428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.889453 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7d3bf0097349fa8255459ed39b4596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.893781 11411 tablet_bootstrap.cc:492] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.895171 11411 tablet_bootstrap.cc:654] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.897617 11411 tablet_bootstrap.cc:492] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.897744 11411 ts_tablet_manager.cc:1397] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.898248 11411 raft_consensus.cc:348] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.898409 11411 raft_consensus.cc:374] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.898464 11411 raft_consensus.cc:729] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.898700 11411 consensus_queue.cc:260] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.898844 11411 raft_consensus.cc:388] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.898924 11411 raft_consensus.cc:482] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.898998 11411 raft_consensus.cc:3037] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.901002 11411 raft_consensus.cc:504] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.901268 11411 leader_election.cc:302] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.901541 11411 leader_election.cc:288] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.901597 11413 raft_consensus.cc:2781] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.901783 11413 raft_consensus.cc:686] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.902045 11413 consensus_queue.cc:237] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.902415 11411 ts_tablet_manager.cc:1428] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:52.906975 11112 catalog_manager.cc:5495] T 2f7d3bf0097349fa8255459ed39b4596 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:52.940866 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44916:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:52.951678 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 847977f89a3e4f3e9342654a7ee6350d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.82 [id=9c8eb582978c4f53823f74cba8cb074a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:52.952104 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847977f89a3e4f3e9342654a7ee6350d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:52.956826 11411 tablet_bootstrap.cc:492] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:52.958392 11411 tablet_bootstrap.cc:654] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:52.960974 11411 tablet_bootstrap.cc:492] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:52.961090 11411 ts_tablet_manager.cc:1397] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:52.961725 11411 raft_consensus.cc:348] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.961880 11411 raft_consensus.cc:374] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:52.961947 11411 raft_consensus.cc:729] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:52.962159 11411 consensus_queue.cc:260] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.962316 11411 raft_consensus.cc:388] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:52.962385 11411 raft_consensus.cc:482] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:52.962463 11411 raft_consensus.cc:3037] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:52.964334 11411 raft_consensus.cc:504] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.964624 11411 leader_election.cc:302] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:52.964833 11411 leader_election.cc:288] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:52.964903 11413 raft_consensus.cc:2781] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:52.965126 11413 raft_consensus.cc:686] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:52.965423 11411 ts_tablet_manager.cc:1428] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:52.965365 11413 consensus_queue.cc:237] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:52.969828 11112 catalog_manager.cc:5495] T 847977f89a3e4f3e9342654a7ee6350d P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.002239 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44928:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.012856 11297 tablet_service.cc:1467] Processing CreateTablet for tablet e6988abb46db4fddb7b126866623a32a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.83 [id=c157419b38cb475d814b7581cd3c820f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.013451 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6988abb46db4fddb7b126866623a32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.017925 11411 tablet_bootstrap.cc:492] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.019331 11411 tablet_bootstrap.cc:654] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.021821 11411 tablet_bootstrap.cc:492] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.021950 11411 ts_tablet_manager.cc:1397] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:53.022532 11411 raft_consensus.cc:348] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.022723 11411 raft_consensus.cc:374] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.022778 11411 raft_consensus.cc:729] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.022938 11411 consensus_queue.cc:260] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.023056 11411 raft_consensus.cc:388] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.023123 11411 raft_consensus.cc:482] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.023185 11411 raft_consensus.cc:3037] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.025010 11411 raft_consensus.cc:504] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.025203 11411 leader_election.cc:302] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.025432 11411 leader_election.cc:288] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.025501 11413 raft_consensus.cc:2781] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.025694 11413 raft_consensus.cc:686] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.025991 11411 ts_tablet_manager.cc:1428] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:53.025942 11413 consensus_queue.cc:237] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.032117 11112 catalog_manager.cc:5495] T e6988abb46db4fddb7b126866623a32a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.067307 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44938:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.078310 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 06cfa6372b524c389f0db805c758878a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.84 [id=690325ca8d2641d69dac87b750430b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.078758 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cfa6372b524c389f0db805c758878a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.083001 11411 tablet_bootstrap.cc:492] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.084278 11411 tablet_bootstrap.cc:654] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.087541 11411 tablet_bootstrap.cc:492] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.087705 11411 ts_tablet_manager.cc:1397] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:53.088339 11411 raft_consensus.cc:348] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.088479 11411 raft_consensus.cc:374] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.088577 11411 raft_consensus.cc:729] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.088824 11411 consensus_queue.cc:260] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.089126 11411 raft_consensus.cc:388] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.089231 11411 raft_consensus.cc:482] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.089375 11411 raft_consensus.cc:3037] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.092011 11411 raft_consensus.cc:504] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.092301 11411 leader_election.cc:302] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.092545 11411 leader_election.cc:288] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.092651 11413 raft_consensus.cc:2781] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.092896 11413 raft_consensus.cc:686] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.093148 11413 consensus_queue.cc:237] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.093511 11411 ts_tablet_manager.cc:1428] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 06:39:53.098872 11113 catalog_manager.cc:5495] T 06cfa6372b524c389f0db805c758878a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.113801 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:53.123946 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:53.127525 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.011s sys 0.000s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":155,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1227,"drs_written":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4,"lbm_write_time_us":743,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.128667 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 599 bytes on disk
I20260809 06:39:53.129810 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.130446 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252423
I20260809 06:39:53.132946 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44940:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.142498 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.013s sys 0.000s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1059,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1074,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.143692 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 599 bytes on disk
I20260809 06:39:53.145241 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a6719cbd230c4702a9e4898d1eabf431 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.85 [id=7ada8bc4245446c0b0fd2f37d17eda34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.145386 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.145722 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6719cbd230c4702a9e4898d1eabf431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.149677 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252423
I20260809 06:39:53.150519 11411 tablet_bootstrap.cc:492] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.151811 11411 tablet_bootstrap.cc:654] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.155288 11411 tablet_bootstrap.cc:492] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.155444 11411 ts_tablet_manager.cc:1397] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:53.156131 11411 raft_consensus.cc:348] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.156353 11411 raft_consensus.cc:374] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.156428 11411 raft_consensus.cc:729] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.156646 11411 consensus_queue.cc:260] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.156803 11411 raft_consensus.cc:388] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.156895 11411 raft_consensus.cc:482] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.156984 11411 raft_consensus.cc:3037] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.158934 11411 raft_consensus.cc:504] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.159196 11411 leader_election.cc:302] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.159447 11413 raft_consensus.cc:2781] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.159591 11413 raft_consensus.cc:686] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.159860 11413 consensus_queue.cc:237] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.161934 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.026s sys 0.003s Metrics: {"bytes_written":42742,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32429,"cfile_cache_miss":31,"cfile_cache_miss_bytes":48193,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":718,"dirs.run_cpu_time_us":835,"dirs.run_wall_time_us":4553,"drs_written":1,"lbm_read_time_us":1337,"lbm_reads_lt_1ms":63,"lbm_write_time_us":811,"lbm_writes_lt_1ms":28,"mutex_wait_us":552,"num_input_rowsets":2,"peak_mem_usage":22856,"rows_written":171,"spinlock_wait_cycles":417024}
I20260809 06:39:53.162902 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260809 06:39:53.164830 11411 leader_election.cc:288] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.165617 11411 ts_tablet_manager.cc:1428] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260809 06:39:53.165974 11113 catalog_manager.cc:5495] T a6719cbd230c4702a9e4898d1eabf431 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.177428 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.000s Metrics: {"bytes_written":329,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":37,"dirs.run_cpu_time_us":150,"dirs.run_wall_time_us":3478,"drs_written":1,"lbm_read_time_us":509,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1003,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:53.178462 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.456311
I20260809 06:39:53.188596 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":571,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":137,"lbm_read_time_us":510,"lbm_reads_lt_1ms":25,"lbm_write_time_us":373,"lbm_writes_lt_1ms":9,"reinsert_count":0,"spinlock_wait_cycles":13696,"update_count":1}
I20260809 06:39:53.189860 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7557 bytes on disk
I20260809 06:39:53.193427 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":160,"lbm_reads_lt_1ms":8}
I20260809 06:39:53.193101 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.020s sys 0.009s Metrics: {"bytes_written":42742,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32429,"cfile_cache_miss":31,"cfile_cache_miss_bytes":48193,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":531,"dirs.run_cpu_time_us":727,"dirs.run_wall_time_us":5686,"drs_written":1,"lbm_read_time_us":1383,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1110,"lbm_writes_lt_1ms":28,"mutex_wait_us":54,"num_input_rowsets":2,"peak_mem_usage":22856,"rows_written":171,"spinlock_wait_cycles":7936}
I20260809 06:39:53.194221 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.220950
I20260809 06:39:53.194407 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7322 bytes on disk
I20260809 06:39:53.196000 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.196631 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000015
I20260809 06:39:53.208818 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.000s Metrics: {"bytes_written":5193,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":133,"dirs.run_wall_time_us":1071,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":966,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:53.209705 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.221021
I20260809 06:39:53.225626 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44952:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.227489 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.029s sys 0.003s Metrics: {"bytes_written":43187,"cfile_cache_hit":13,"cfile_cache_hit_bytes":32842,"cfile_cache_miss":31,"cfile_cache_miss_bytes":48576,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1252,"dirs.run_cpu_time_us":974,"dirs.run_wall_time_us":6982,"drs_written":1,"lbm_read_time_us":1406,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1284,"lbm_writes_lt_1ms":28,"mutex_wait_us":52,"num_input_rowsets":2,"peak_mem_usage":23076,"rows_written":173,"thread_start_us":754,"threads_started":1}
I20260809 06:39:53.228613 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7423 bytes on disk
I20260809 06:39:53.229174 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.239723 11297 tablet_service.cc:1467] Processing CreateTablet for tablet e1143982c6484b8e80e14101eca7f189 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.86 [id=4845a570a5754c52aa0da4691807d989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.240247 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1143982c6484b8e80e14101eca7f189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.240650 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.027s sys 0.000s Metrics: {"bytes_written":43187,"cfile_cache_hit":12,"cfile_cache_hit_bytes":32812,"cfile_cache_miss":28,"cfile_cache_miss_bytes":48448,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":492,"dirs.run_cpu_time_us":461,"dirs.run_wall_time_us":2069,"drs_written":1,"lbm_read_time_us":1162,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1023,"lbm_writes_lt_1ms":28,"mutex_wait_us":68,"num_input_rowsets":2,"peak_mem_usage":23268,"rows_written":173}
I20260809 06:39:53.241550 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7423 bytes on disk
I20260809 06:39:53.242113 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.242650 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:53.245994 11411 tablet_bootstrap.cc:492] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.247504 11411 tablet_bootstrap.cc:654] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.250300 11411 tablet_bootstrap.cc:492] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.250452 11411 ts_tablet_manager.cc:1397] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:53.251066 11411 raft_consensus.cc:348] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.251242 11411 raft_consensus.cc:374] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.251353 11411 raft_consensus.cc:729] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.251631 11411 consensus_queue.cc:260] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.251874 11411 raft_consensus.cc:388] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.251986 11411 raft_consensus.cc:482] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.252112 11411 raft_consensus.cc:3037] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.252373 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":1016,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":672,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:53.253314 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.224352
I20260809 06:39:53.254602 11411 raft_consensus.cc:504] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.254946 11411 leader_election.cc:302] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.255193 11411 leader_election.cc:288] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.255290 11413 raft_consensus.cc:2781] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.255479 11413 raft_consensus.cc:686] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.255718 11413 consensus_queue.cc:237] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.256044 11411 ts_tablet_manager.cc:1428] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:53.261343 11113 catalog_manager.cc:5495] T e1143982c6484b8e80e14101eca7f189 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.288158 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.025s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":33147,"cfile_cache_miss":28,"cfile_cache_miss_bytes":48699,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":3747,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":1355,"drs_written":1,"lbm_read_time_us":1208,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1215,"lbm_writes_lt_1ms":32,"mutex_wait_us":39,"num_input_rowsets":2,"peak_mem_usage":23348,"reinsert_count":0,"rows_written":175,"spinlock_wait_cycles":1280,"update_count":1}
I20260809 06:39:53.289494 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7491 bytes on disk
I20260809 06:39:53.290011 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.300843 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44956:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.312170 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c0d7aff90a4e48fc85a36afdb0221010 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.87 [id=0ff253f05cf34af89b4e62680f8e1cda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.312681 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d7aff90a4e48fc85a36afdb0221010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.317181 11411 tablet_bootstrap.cc:492] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.318559 11411 tablet_bootstrap.cc:654] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.320986 11411 tablet_bootstrap.cc:492] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.321166 11411 ts_tablet_manager.cc:1397] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:53.321841 11411 raft_consensus.cc:348] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.322122 11411 raft_consensus.cc:374] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.322221 11411 raft_consensus.cc:729] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.322482 11411 consensus_queue.cc:260] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.322629 11411 raft_consensus.cc:388] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.322775 11411 raft_consensus.cc:482] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.322876 11411 raft_consensus.cc:3037] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.324621 11411 raft_consensus.cc:504] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.324862 11411 leader_election.cc:302] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.325080 11411 leader_election.cc:288] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.325138 11413 raft_consensus.cc:2781] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.325330 11413 raft_consensus.cc:686] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.325641 11411 ts_tablet_manager.cc:1428] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:53.325608 11413 consensus_queue.cc:237] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.331540 11112 catalog_manager.cc:5495] T c0d7aff90a4e48fc85a36afdb0221010 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.364581 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44964:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.375254 11297 tablet_service.cc:1467] Processing CreateTablet for tablet cadab4f91e7d4856999bc13906962b26 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.88 [id=f95ae8bc68bb43849717bc0d9abdc654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.375772 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cadab4f91e7d4856999bc13906962b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.380353 11411 tablet_bootstrap.cc:492] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.381855 11411 tablet_bootstrap.cc:654] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.384219 11411 tablet_bootstrap.cc:492] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.384351 11411 ts_tablet_manager.cc:1397] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:53.385010 11411 raft_consensus.cc:348] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.385228 11411 raft_consensus.cc:374] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.385378 11411 raft_consensus.cc:729] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.385636 11411 consensus_queue.cc:260] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.385882 11411 raft_consensus.cc:388] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.385984 11411 raft_consensus.cc:482] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.386160 11411 raft_consensus.cc:3037] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.388026 11411 raft_consensus.cc:504] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.388263 11411 leader_election.cc:302] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.388453 11411 leader_election.cc:288] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.388514 11413 raft_consensus.cc:2781] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.388685 11413 raft_consensus.cc:686] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.389134 11411 ts_tablet_manager.cc:1428] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:53.388967 11413 consensus_queue.cc:237] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.395506 11113 catalog_manager.cc:5495] T cadab4f91e7d4856999bc13906962b26 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.429961 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44976:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.441386 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 0c575dc197e441e48373b589ec063aff (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.89 [id=b37b953f8e484da59bcdf0371acf0e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.441952 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c575dc197e441e48373b589ec063aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.447489 11411 tablet_bootstrap.cc:492] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.449065 11411 tablet_bootstrap.cc:654] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.451869 11411 tablet_bootstrap.cc:492] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.452044 11411 ts_tablet_manager.cc:1397] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:53.452790 11411 raft_consensus.cc:348] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.453023 11411 raft_consensus.cc:374] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.453102 11411 raft_consensus.cc:729] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.453323 11411 consensus_queue.cc:260] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.453544 11411 raft_consensus.cc:388] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.453656 11411 raft_consensus.cc:482] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.453744 11411 raft_consensus.cc:3037] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.456663 11411 raft_consensus.cc:504] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.456928 11411 leader_election.cc:302] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.457144 11411 leader_election.cc:288] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.457240 11413 raft_consensus.cc:2781] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.457432 11413 raft_consensus.cc:686] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.457623 11411 ts_tablet_manager.cc:1428] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 06:39:53.457702 11413 consensus_queue.cc:237] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.463564 11113 catalog_manager.cc:5495] T 0c575dc197e441e48373b589ec063aff P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.480340 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260809 06:39:53.491870 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.003s sys 0.006s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":1477,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":622,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.492738 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 597 bytes on disk
I20260809 06:39:53.493572 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.495270 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.252198
I20260809 06:39:53.500003 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44992:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.512953 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 22e43fa3cfbd436dbed9db317eb69038 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.90 [id=fdbf3d1238984bf49f2750b252745483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.513474 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e43fa3cfbd436dbed9db317eb69038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.519958 11411 tablet_bootstrap.cc:492] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.521387 11411 tablet_bootstrap.cc:654] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.524101 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.016s sys 0.011s Metrics: {"bytes_written":44925,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34277,"cfile_cache_miss":28,"cfile_cache_miss_bytes":50549,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":567,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1269,"drs_written":1,"lbm_read_time_us":983,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1166,"lbm_writes_lt_1ms":28,"mutex_wait_us":145,"num_input_rowsets":2,"peak_mem_usage":24276,"rows_written":181,"spinlock_wait_cycles":6144}
I20260809 06:39:53.524518 11411 tablet_bootstrap.cc:492] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.524726 11411 ts_tablet_manager.cc:1397] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:53.524988 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7757 bytes on disk
I20260809 06:39:53.525483 11411 raft_consensus.cc:348] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.525677 11411 raft_consensus.cc:374] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.525578 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.525811 11411 raft_consensus.cc:729] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.526046 11411 consensus_queue.cc:260] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.526309 11411 raft_consensus.cc:388] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.526304 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260809 06:39:53.526397 11411 raft_consensus.cc:482] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.526512 11411 raft_consensus.cc:3037] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.528329 11411 raft_consensus.cc:504] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.528590 11411 leader_election.cc:302] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.528847 11411 leader_election.cc:288] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.528925 11413 raft_consensus.cc:2781] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.529086 11413 raft_consensus.cc:686] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.529419 11411 ts_tablet_manager.cc:1428] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:53.529307 11413 consensus_queue.cc:237] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.534874 11113 catalog_manager.cc:5495] T 22e43fa3cfbd436dbed9db317eb69038 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.538044 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.004s sys 0.006s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":89,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":944,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":711,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:53.538920 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.161196
I20260809 06:39:53.541043 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:53.553869 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":6490,"cfile_init":1,"dirs.queue_time_us":315,"dirs.run_cpu_time_us":232,"dirs.run_wall_time_us":1015,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":640,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.554976 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 594 bytes on disk
I20260809 06:39:53.555721 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.556489 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.250326
I20260809 06:39:53.572376 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.028s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":34619,"cfile_cache_miss":28,"cfile_cache_miss_bytes":50843,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":733,"dirs.run_cpu_time_us":284,"dirs.run_wall_time_us":1472,"drs_written":1,"lbm_read_time_us":1232,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1471,"lbm_writes_lt_1ms":32,"mutex_wait_us":178,"num_input_rowsets":2,"peak_mem_usage":24364,"reinsert_count":0,"rows_written":183,"spinlock_wait_cycles":4480,"update_count":1}
I20260809 06:39:53.573705 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7820 bytes on disk
I20260809 06:39:53.574324 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.586704 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44998:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.588369 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.031s sys 0.000s Metrics: {"bytes_written":45350,"cfile_cache_hit":13,"cfile_cache_hit_bytes":34637,"cfile_cache_miss":31,"cfile_cache_miss_bytes":51435,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":536,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1010,"drs_written":1,"lbm_read_time_us":1366,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1048,"lbm_writes_lt_1ms":28,"mutex_wait_us":51,"num_input_rowsets":2,"peak_mem_usage":24583,"rows_written":183,"spinlock_wait_cycles":896}
I20260809 06:39:53.589404 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7833 bytes on disk
I20260809 06:39:53.590008 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.599231 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 457f8d5a88844353a440c0a1c0006a93 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.91 [id=3a85a395ea7242ab81b193f2f5918cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.599679 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457f8d5a88844353a440c0a1c0006a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.603878 11411 tablet_bootstrap.cc:492] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.605286 11411 tablet_bootstrap.cc:654] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.607824 11411 tablet_bootstrap.cc:492] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.607954 11411 ts_tablet_manager.cc:1397] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:53.608513 11411 raft_consensus.cc:348] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.608690 11411 raft_consensus.cc:374] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.608744 11411 raft_consensus.cc:729] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.608911 11411 consensus_queue.cc:260] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.609061 11411 raft_consensus.cc:388] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.609134 11411 raft_consensus.cc:482] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.609200 11411 raft_consensus.cc:3037] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.611491 11411 raft_consensus.cc:504] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.611680 11411 leader_election.cc:302] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.611866 11411 leader_election.cc:288] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.611931 11413 raft_consensus.cc:2781] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.612133 11413 raft_consensus.cc:686] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.612460 11411 ts_tablet_manager.cc:1428] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 06:39:53.612388 11413 consensus_queue.cc:237] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.617437 11113 catalog_manager.cc:5495] T 457f8d5a88844353a440c0a1c0006a93 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.653852 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45012:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.665634 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a002ec77f1a14ab38b2b6b727695a49b (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.92 [id=e922f392774c4c6ca7286995f53bd809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.666162 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a002ec77f1a14ab38b2b6b727695a49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.670928 11411 tablet_bootstrap.cc:492] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.672274 11411 tablet_bootstrap.cc:654] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.674522 11411 tablet_bootstrap.cc:492] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.674638 11411 ts_tablet_manager.cc:1397] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:53.675189 11411 raft_consensus.cc:348] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.675334 11411 raft_consensus.cc:374] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.675402 11411 raft_consensus.cc:729] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.675628 11411 consensus_queue.cc:260] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.675791 11411 raft_consensus.cc:388] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.675848 11411 raft_consensus.cc:482] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.675927 11411 raft_consensus.cc:3037] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.677747 11411 raft_consensus.cc:504] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.677950 11411 leader_election.cc:302] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.678123 11411 leader_election.cc:288] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.678225 11413 raft_consensus.cc:2781] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.678413 11413 raft_consensus.cc:686] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.678732 11411 ts_tablet_manager.cc:1428] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:53.678655 11413 consensus_queue.cc:237] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.684088 11113 catalog_manager.cc:5495] T a002ec77f1a14ab38b2b6b727695a49b P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.720238 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45026:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.732636 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 5323dcc60b0f4fed9b48f7068353efd8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.93 [id=6dfc11cea9464c9e9af6f176864fb047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.733167 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5323dcc60b0f4fed9b48f7068353efd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.737941 11411 tablet_bootstrap.cc:492] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.739630 11411 tablet_bootstrap.cc:654] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.742476 11411 tablet_bootstrap.cc:492] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.742654 11411 ts_tablet_manager.cc:1397] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:53.743431 11411 raft_consensus.cc:348] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.743673 11411 raft_consensus.cc:374] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.743781 11411 raft_consensus.cc:729] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.744057 11411 consensus_queue.cc:260] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.744254 11411 raft_consensus.cc:388] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.744380 11411 raft_consensus.cc:482] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.744489 11411 raft_consensus.cc:3037] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.746425 11411 raft_consensus.cc:504] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.746671 11411 leader_election.cc:302] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.746884 11411 leader_election.cc:288] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.746946 11413 raft_consensus.cc:2781] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.747129 11413 raft_consensus.cc:686] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.747460 11411 ts_tablet_manager.cc:1428] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:53.747375 11413 consensus_queue.cc:237] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.752525 11113 catalog_manager.cc:5495] T 5323dcc60b0f4fed9b48f7068353efd8 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.786829 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45034:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.798461 11297 tablet_service.cc:1467] Processing CreateTablet for tablet ed30fd940e1041bea47314f59c148eda (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.94 [id=667f723508af439cb5a7cd2759354026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.798931 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed30fd940e1041bea47314f59c148eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.803488 11411 tablet_bootstrap.cc:492] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.804850 11411 tablet_bootstrap.cc:654] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.807101 11411 tablet_bootstrap.cc:492] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.807263 11411 ts_tablet_manager.cc:1397] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:53.807888 11411 raft_consensus.cc:348] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.808073 11411 raft_consensus.cc:374] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.808192 11411 raft_consensus.cc:729] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.808434 11411 consensus_queue.cc:260] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.808655 11411 raft_consensus.cc:388] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.808753 11411 raft_consensus.cc:482] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.808871 11411 raft_consensus.cc:3037] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.810791 11411 raft_consensus.cc:504] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.811007 11411 leader_election.cc:302] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.811183 11411 leader_election.cc:288] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.811260 11413 raft_consensus.cc:2781] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.811415 11413 raft_consensus.cc:686] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.811748 11411 ts_tablet_manager.cc:1428] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:53.811664 11413 consensus_queue.cc:237] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.817382 11113 catalog_manager.cc:5495] T ed30fd940e1041bea47314f59c148eda P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.825245 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:53.837584 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.001s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":85,"dirs.run_cpu_time_us":188,"dirs.run_wall_time_us":1164,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":764,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.838363 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248078
I20260809 06:39:53.840938 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:53.854079 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.003s Metrics: {"bytes_written":6491,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1140,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":804,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:53.855103 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 595 bytes on disk
I20260809 06:39:53.855991 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":165,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.856927 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248403
I20260809 06:39:53.866530 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.026s sys 0.001s Metrics: {"bytes_written":47096,"cfile_cache_hit":13,"cfile_cache_hit_bytes":36109,"cfile_cache_miss":31,"cfile_cache_miss_bytes":53571,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":659,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":852,"drs_written":1,"lbm_read_time_us":1118,"lbm_reads_lt_1ms":67,"lbm_write_time_us":723,"lbm_writes_lt_1ms":28,"mutex_wait_us":204,"num_input_rowsets":2,"peak_mem_usage":25599,"rows_written":191}
I20260809 06:39:53.867936 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8168 bytes on disk
I20260809 06:39:53.868775 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.872558 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45044:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.885442 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 931136d71bca47d097e7062c32ee10e6 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.95 [id=056c2191e91244b3a772eab611ac144d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.885957 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931136d71bca47d097e7062c32ee10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.886022 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.028s sys 0.000s Metrics: {"bytes_written":47096,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36112,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53204,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":850,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1035,"drs_written":1,"lbm_read_time_us":1181,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1008,"lbm_writes_lt_1ms":28,"mutex_wait_us":45,"num_input_rowsets":2,"peak_mem_usage":25546,"rows_written":191,"spinlock_wait_cycles":768}
I20260809 06:39:53.886837 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:53.890947 11411 tablet_bootstrap.cc:492] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.892501 11411 tablet_bootstrap.cc:654] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.895361 11411 tablet_bootstrap.cc:492] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.895538 11411 ts_tablet_manager.cc:1397] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:53.896118 11411 raft_consensus.cc:348] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.896319 11411 raft_consensus.cc:374] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.896445 11411 raft_consensus.cc:729] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.896692 11411 consensus_queue.cc:260] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.896920 11411 raft_consensus.cc:388] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.897034 11411 raft_consensus.cc:482] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.896893 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":229,"dirs.run_wall_time_us":960,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":626,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2,"spinlock_wait_cycles":29824}
I20260809 06:39:53.897156 11411 raft_consensus.cc:3037] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.897656 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8403 bytes on disk
I20260809 06:39:53.898237 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":102,"lbm_reads_lt_1ms":8}
I20260809 06:39:53.898789 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.198113
I20260809 06:39:53.899135 11411 raft_consensus.cc:504] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.899374 11411 leader_election.cc:302] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.899592 11411 leader_election.cc:288] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.899679 11413 raft_consensus.cc:2781] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.899845 11413 raft_consensus.cc:686] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.900113 11411 ts_tablet_manager.cc:1428] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:53.900144 11413 consensus_queue.cc:237] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.907590 11113 catalog_manager.cc:5495] T 931136d71bca47d097e7062c32ee10e6 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:53.927531 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.028s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":36454,"cfile_cache_miss":28,"cfile_cache_miss_bytes":53512,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":700,"dirs.run_cpu_time_us":186,"dirs.run_wall_time_us":1118,"drs_written":1,"lbm_read_time_us":960,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1058,"lbm_writes_lt_1ms":32,"mutex_wait_us":122,"num_input_rowsets":2,"peak_mem_usage":25634,"reinsert_count":0,"rows_written":193,"update_count":1}
I20260809 06:39:53.928261 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8238 bytes on disk
I20260809 06:39:53.928838 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260809 06:39:53.945907 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45052:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:53.957149 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef4b355fa874e7c86955d2b875e4535 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.96 [id=5c9ae6e0e2224641b24a21cc85d56953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:53.957646 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef4b355fa874e7c86955d2b875e4535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:53.962301 11411 tablet_bootstrap.cc:492] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:53.963599 11411 tablet_bootstrap.cc:654] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:53.966171 11411 tablet_bootstrap.cc:492] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:53.966339 11411 ts_tablet_manager.cc:1397] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:53.966908 11411 raft_consensus.cc:348] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.967089 11411 raft_consensus.cc:374] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:53.967226 11411 raft_consensus.cc:729] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:53.967510 11411 consensus_queue.cc:260] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.967778 11411 raft_consensus.cc:388] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:53.967872 11411 raft_consensus.cc:482] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:53.967976 11411 raft_consensus.cc:3037] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:53.970058 11411 raft_consensus.cc:504] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.970314 11411 leader_election.cc:302] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:53.970515 11411 leader_election.cc:288] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:53.970582 11413 raft_consensus.cc:2781] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:53.970821 11413 raft_consensus.cc:686] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:53.970988 11411 ts_tablet_manager.cc:1428] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:53.971076 11413 consensus_queue.cc:237] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:53.975906 11113 catalog_manager.cc:5495] T 7ef4b355fa874e7c86955d2b875e4535 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.010612 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45068:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.021618 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 538abd2eb1d04bf989b87c9b35357239 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.97 [id=1d88e3679ae34bc2b45323d2183754a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.022162 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538abd2eb1d04bf989b87c9b35357239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.026646 11411 tablet_bootstrap.cc:492] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.028108 11411 tablet_bootstrap.cc:654] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.031031 11411 tablet_bootstrap.cc:492] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.031198 11411 ts_tablet_manager.cc:1397] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:54.031880 11411 raft_consensus.cc:348] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.032120 11411 raft_consensus.cc:374] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.032241 11411 raft_consensus.cc:729] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.032485 11411 consensus_queue.cc:260] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.032687 11411 raft_consensus.cc:388] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.032816 11411 raft_consensus.cc:482] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.032929 11411 raft_consensus.cc:3037] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.034747 11411 raft_consensus.cc:504] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.034991 11411 leader_election.cc:302] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.035218 11411 leader_election.cc:288] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.035291 11413 raft_consensus.cc:2781] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.035465 11413 raft_consensus.cc:686] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.035743 11413 consensus_queue.cc:237] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.036062 11411 ts_tablet_manager.cc:1428] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:54.042477 11112 catalog_manager.cc:5495] T 538abd2eb1d04bf989b87c9b35357239 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.077128 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45082:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.087993 11297 tablet_service.cc:1467] Processing CreateTablet for tablet ae0a40188559489586bfbd829c1fe8d4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.98 [id=46d2d85285694efa8d9ae704d4bdbcf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.088548 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0a40188559489586bfbd829c1fe8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.093178 11411 tablet_bootstrap.cc:492] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.094552 11411 tablet_bootstrap.cc:654] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.097235 11411 tablet_bootstrap.cc:492] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.097420 11411 ts_tablet_manager.cc:1397] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:54.098043 11411 raft_consensus.cc:348] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.098184 11411 raft_consensus.cc:374] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.098300 11411 raft_consensus.cc:729] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.098523 11411 consensus_queue.cc:260] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.098670 11411 raft_consensus.cc:388] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.098719 11411 raft_consensus.cc:482] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.098809 11411 raft_consensus.cc:3037] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.100662 11411 raft_consensus.cc:504] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.100860 11411 leader_election.cc:302] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.101063 11411 leader_election.cc:288] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.101125 11413 raft_consensus.cc:2781] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.101305 11413 raft_consensus.cc:686] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.101544 11413 consensus_queue.cc:237] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.101812 11411 ts_tablet_manager.cc:1428] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:54.106964 11112 catalog_manager.cc:5495] T ae0a40188559489586bfbd829c1fe8d4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.119755 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000078
I20260809 06:39:54.132244 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.004s Metrics: {"bytes_written":6492,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":1135,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":868,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:54.133026 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.246339
I20260809 06:39:54.151048 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45084:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.162635 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.025s sys 0.003s Metrics: {"bytes_written":48837,"cfile_cache_hit":12,"cfile_cache_hit_bytes":37584,"cfile_cache_miss":28,"cfile_cache_miss_bytes":55364,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":425,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":1225,"drs_written":1,"lbm_read_time_us":1202,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1016,"lbm_writes_lt_1ms":28,"mutex_wait_us":44,"num_input_rowsets":2,"peak_mem_usage":26562,"rows_written":199,"spinlock_wait_cycles":6912}
I20260809 06:39:54.164180 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:54.164736 11297 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ad9f0ff3146b89d523856b291067a (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.99 [id=0af8261d1c2947a3b0432456ca9d4e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.165246 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ad9f0ff3146b89d523856b291067a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.170171 11411 tablet_bootstrap.cc:492] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.171998 11411 tablet_bootstrap.cc:654] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.174657 11411 tablet_bootstrap.cc:492] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.174830 11411 ts_tablet_manager.cc:1397] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:54.175741 11411 raft_consensus.cc:348] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.175981 11411 raft_consensus.cc:374] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.176102 11411 raft_consensus.cc:729] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.176352 11411 consensus_queue.cc:260] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.176584 11411 raft_consensus.cc:388] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.176699 11411 raft_consensus.cc:482] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.176797 11411 raft_consensus.cc:3037] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.179394 11411 raft_consensus.cc:504] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.179662 11411 leader_election.cc:302] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.179716 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000079
I20260809 06:39:54.179908 11411 leader_election.cc:288] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.179991 11413 raft_consensus.cc:2781] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.180178 11413 raft_consensus.cc:686] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.180419 11411 ts_tablet_manager.cc:1428] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:54.181617 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":5156,"cfile_init":1,"dirs.queue_time_us":46,"dirs.run_cpu_time_us":187,"dirs.run_wall_time_us":1160,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":853,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:54.180426 11413 consensus_queue.cc:237] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.182562 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.223468
I20260809 06:39:54.189051 11112 catalog_manager.cc:5495] T cb9ad9f0ff3146b89d523856b291067a P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.193215 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.008s sys 0.003s Metrics: {"bytes_written":6455,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":4561,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":708,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:54.194290 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.246022
I20260809 06:39:54.228830 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.031s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":37917,"cfile_cache_miss":31,"cfile_cache_miss_bytes":56217,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":718,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1176,"drs_written":1,"lbm_read_time_us":1367,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1209,"lbm_writes_lt_1ms":32,"mutex_wait_us":87,"num_input_rowsets":2,"peak_mem_usage":26700,"reinsert_count":0,"rows_written":201,"update_count":1}
I20260809 06:39:54.229678 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8567 bytes on disk
I20260809 06:39:54.230266 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.230736 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.043s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":37926,"cfile_cache_miss":28,"cfile_cache_miss_bytes":55658,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":526,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":1070,"drs_written":1,"lbm_read_time_us":1506,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1186,"lbm_writes_lt_1ms":32,"mutex_wait_us":101,"num_input_rowsets":2,"peak_mem_usage":26650,"reinsert_count":0,"rows_written":201,"update_count":1}
I20260809 06:39:54.231689 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8567 bytes on disk
I20260809 06:39:54.232339 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":134,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.243979 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45088:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.255513 11297 tablet_service.cc:1467] Processing CreateTablet for tablet edbb4adc1b124cb1ac88d0b3dc6980a0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.100 [id=70a28fb6e23249f4a494aedba5530e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.255991 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edbb4adc1b124cb1ac88d0b3dc6980a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.262589 11411 tablet_bootstrap.cc:492] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.264473 11411 tablet_bootstrap.cc:654] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.268596 11411 tablet_bootstrap.cc:492] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.268723 11411 ts_tablet_manager.cc:1397] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 06:39:54.269413 11411 raft_consensus.cc:348] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.269655 11411 raft_consensus.cc:374] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.269767 11411 raft_consensus.cc:729] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.270015 11411 consensus_queue.cc:260] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.270262 11411 raft_consensus.cc:388] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.270371 11411 raft_consensus.cc:482] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.270488 11411 raft_consensus.cc:3037] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.272922 11411 raft_consensus.cc:504] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.273217 11411 leader_election.cc:302] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.273433 11411 leader_election.cc:288] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.273527 11413 raft_consensus.cc:2781] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.273729 11413 raft_consensus.cc:686] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.273931 11411 ts_tablet_manager.cc:1428] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:54.273964 11413 consensus_queue.cc:237] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.280210 11112 catalog_manager.cc:5495] T edbb4adc1b124cb1ac88d0b3dc6980a0 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.313316 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45092:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.324441 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b3f12c22b8664af99575c75f9bef09cd (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.101 [id=85c84e12a13a47509e031685a54253a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.324932 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f12c22b8664af99575c75f9bef09cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.329810 11411 tablet_bootstrap.cc:492] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.331281 11411 tablet_bootstrap.cc:654] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.333496 11411 tablet_bootstrap.cc:492] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.333622 11411 ts_tablet_manager.cc:1397] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:54.334172 11411 raft_consensus.cc:348] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.334338 11411 raft_consensus.cc:374] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.334391 11411 raft_consensus.cc:729] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.334551 11411 consensus_queue.cc:260] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.334674 11411 raft_consensus.cc:388] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.334739 11411 raft_consensus.cc:482] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.334810 11411 raft_consensus.cc:3037] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.336853 11411 raft_consensus.cc:504] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.337035 11411 leader_election.cc:302] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.337227 11411 leader_election.cc:288] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.337327 11413 raft_consensus.cc:2781] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.337563 11413 raft_consensus.cc:686] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.337795 11411 ts_tablet_manager.cc:1428] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:54.337819 11413 consensus_queue.cc:237] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.343446 11112 catalog_manager.cc:5495] T b3f12c22b8664af99575c75f9bef09cd P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.377964 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45108:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.389986 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a35930ecb051467c95ee4d05fd682fea (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.102 [id=cc34fbce302b497fb664cb80c8d1756f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.390462 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35930ecb051467c95ee4d05fd682fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.395660 11411 tablet_bootstrap.cc:492] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.397149 11411 tablet_bootstrap.cc:654] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.400096 11411 tablet_bootstrap.cc:492] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.400272 11411 ts_tablet_manager.cc:1397] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:54.400895 11411 raft_consensus.cc:348] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.401060 11411 raft_consensus.cc:374] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.401258 11411 raft_consensus.cc:729] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.401526 11411 consensus_queue.cc:260] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.401729 11411 raft_consensus.cc:388] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.401822 11411 raft_consensus.cc:482] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.401980 11411 raft_consensus.cc:3037] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.403865 11411 raft_consensus.cc:504] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.404142 11411 leader_election.cc:302] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.404414 11411 leader_election.cc:288] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.404484 11413 raft_consensus.cc:2781] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.404685 11413 raft_consensus.cc:686] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.404932 11411 ts_tablet_manager.cc:1428] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:54.404930 11413 consensus_queue.cc:237] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.411043 11112 catalog_manager.cc:5495] T a35930ecb051467c95ee4d05fd682fea P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.446796 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45122:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.460002 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 43d718f9710b4250a6f57225394e4eab (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.103 [id=ca83177885e9467595432a07d64a4b2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.460484 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d718f9710b4250a6f57225394e4eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.465952 11411 tablet_bootstrap.cc:492] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.467288 11411 tablet_bootstrap.cc:654] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.470495 11411 tablet_bootstrap.cc:492] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.470660 11411 ts_tablet_manager.cc:1397] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:54.471314 11411 raft_consensus.cc:348] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.471511 11411 raft_consensus.cc:374] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.471596 11411 raft_consensus.cc:729] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.471871 11411 consensus_queue.cc:260] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.472084 11411 raft_consensus.cc:388] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.472210 11411 raft_consensus.cc:482] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.472307 11411 raft_consensus.cc:3037] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.474350 11411 raft_consensus.cc:504] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.474612 11411 leader_election.cc:302] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.474831 11411 leader_election.cc:288] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.474907 11413 raft_consensus.cc:2781] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.475139 11413 raft_consensus.cc:686] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.475414 11413 consensus_queue.cc:237] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.475565 11411 ts_tablet_manager.cc:1428] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:54.480798 11112 catalog_manager.cc:5495] T 43d718f9710b4250a6f57225394e4eab P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.481272 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:54.483153 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260809 06:39:54.498546 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.012s sys 0.001s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":55,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1148,"drs_written":1,"lbm_read_time_us":128,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1150,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:54.499571 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 587 bytes on disk
I20260809 06:39:54.500231 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":126,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.500758 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248156
I20260809 06:39:54.510136 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.012s sys 0.001s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":40,"dirs.run_cpu_time_us":126,"dirs.run_wall_time_us":877,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1767,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:54.511090 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.248156
I20260809 06:39:54.539572 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.034s sys 0.003s Metrics: {"bytes_written":51026,"cfile_cache_hit":14,"cfile_cache_hit_bytes":39427,"cfile_cache_miss":34,"cfile_cache_miss_bytes":58795,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":2013,"dirs.run_cpu_time_us":1020,"dirs.run_wall_time_us":5419,"drs_written":1,"lbm_read_time_us":1185,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1192,"lbm_writes_lt_1ms":28,"mutex_wait_us":53,"num_input_rowsets":2,"peak_mem_usage":27937,"rows_written":209,"spinlock_wait_cycles":8960,"thread_start_us":1898,"threads_started":1}
I20260809 06:39:54.539918 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45124:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.104"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.540482 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8909 bytes on disk
I20260809 06:39:54.541112 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.549546 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.035s sys 0.001s Metrics: {"bytes_written":51026,"cfile_cache_hit":14,"cfile_cache_hit_bytes":39427,"cfile_cache_miss":34,"cfile_cache_miss_bytes":58795,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":1229,"dirs.run_cpu_time_us":1125,"dirs.run_wall_time_us":7404,"drs_written":1,"lbm_read_time_us":1593,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1168,"lbm_writes_lt_1ms":28,"mutex_wait_us":74,"num_input_rowsets":2,"peak_mem_usage":27937,"rows_written":209,"thread_start_us":1545,"threads_started":1}
I20260809 06:39:54.550347 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:54.553073 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1166dc773112449c96996536eb1d7216 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.104 [id=9597e70b611842af972ca8c6023f6f78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.553649 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1166dc773112449c96996536eb1d7216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.560050 11411 tablet_bootstrap.cc:492] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.561671 11411 tablet_bootstrap.cc:654] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.562430 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.003s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":999,"drs_written":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1100,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:54.563414 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.537705
I20260809 06:39:54.564991 11411 tablet_bootstrap.cc:492] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.565145 11411 ts_tablet_manager.cc:1397] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:54.565801 11411 raft_consensus.cc:348] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.565938 11411 raft_consensus.cc:374] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.565999 11411 raft_consensus.cc:729] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.566210 11411 consensus_queue.cc:260] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.566359 11411 raft_consensus.cc:388] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.566409 11411 raft_consensus.cc:482] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.566480 11411 raft_consensus.cc:3037] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.568291 11411 raft_consensus.cc:504] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.568538 11411 leader_election.cc:302] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.568748 11411 leader_election.cc:288] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.568812 11413 raft_consensus.cc:2781] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.569008 11413 raft_consensus.cc:686] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.569257 11413 consensus_queue.cc:237] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.569705 11411 ts_tablet_manager.cc:1428] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:54.570958 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":459,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":55,"lbm_read_time_us":352,"lbm_reads_lt_1ms":25,"lbm_write_time_us":235,"lbm_writes_lt_1ms":9,"mutex_wait_us":20,"reinsert_count":0,"spinlock_wait_cycles":2432,"update_count":1}
I20260809 06:39:54.571472 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9076 bytes on disk
I20260809 06:39:54.572206 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":187,"lbm_reads_lt_1ms":8}
I20260809 06:39:54.572839 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.186276
I20260809 06:39:54.576510 11112 catalog_manager.cc:5495] T 1166dc773112449c96996536eb1d7216 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.601498 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.028s sys 0.000s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":39809,"cfile_cache_miss":31,"cfile_cache_miss_bytes":58405,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1033,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1171,"drs_written":1,"lbm_read_time_us":1203,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1077,"lbm_writes_lt_1ms":32,"mutex_wait_us":313,"num_input_rowsets":2,"peak_mem_usage":27937,"reinsert_count":0,"rows_written":211,"spinlock_wait_cycles":896,"update_count":1}
I20260809 06:39:54.602563 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8982 bytes on disk
I20260809 06:39:54.603158 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.611914 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45140:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.623027 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 6a592ee098264201a6fc530ca5bd58ab (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.105 [id=0994e992011344539a5b17e07a0b4257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.623476 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a592ee098264201a6fc530ca5bd58ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.627813 11411 tablet_bootstrap.cc:492] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.629106 11411 tablet_bootstrap.cc:654] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.631632 11411 tablet_bootstrap.cc:492] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.631803 11411 ts_tablet_manager.cc:1397] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 06:39:54.632412 11411 raft_consensus.cc:348] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.632611 11411 raft_consensus.cc:374] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.632712 11411 raft_consensus.cc:729] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.632954 11411 consensus_queue.cc:260] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.633121 11411 raft_consensus.cc:388] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.633217 11411 raft_consensus.cc:482] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.633320 11411 raft_consensus.cc:3037] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.635515 11411 raft_consensus.cc:504] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.635752 11411 leader_election.cc:302] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.635962 11411 leader_election.cc:288] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.636020 11413 raft_consensus.cc:2781] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.636189 11413 raft_consensus.cc:686] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.636420 11413 consensus_queue.cc:237] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.636852 11411 ts_tablet_manager.cc:1428] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:54.642041 11113 catalog_manager.cc:5495] T 6a592ee098264201a6fc530ca5bd58ab P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.683930 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45154:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.106"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.695495 11297 tablet_service.cc:1467] Processing CreateTablet for tablet f09f83d99a284f689ca658d378380bf9 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.106 [id=bf1411ba080445ada8f5f1c9d4cb5772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.695892 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f09f83d99a284f689ca658d378380bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.700143 11411 tablet_bootstrap.cc:492] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.701709 11411 tablet_bootstrap.cc:654] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.704082 11411 tablet_bootstrap.cc:492] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.704246 11411 ts_tablet_manager.cc:1397] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:54.704874 11411 raft_consensus.cc:348] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.705051 11411 raft_consensus.cc:374] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.705152 11411 raft_consensus.cc:729] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.705410 11411 consensus_queue.cc:260] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.705675 11411 raft_consensus.cc:388] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.705768 11411 raft_consensus.cc:482] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.705916 11411 raft_consensus.cc:3037] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.708329 11411 raft_consensus.cc:504] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.708623 11411 leader_election.cc:302] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.708858 11411 leader_election.cc:288] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.708925 11413 raft_consensus.cc:2781] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.709080 11413 raft_consensus.cc:686] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.709439 11411 ts_tablet_manager.cc:1428] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 06:39:54.709309 11413 consensus_queue.cc:237] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.715330 11112 catalog_manager.cc:5495] T f09f83d99a284f689ca658d378380bf9 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.747220 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45166:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.758457 11297 tablet_service.cc:1467] Processing CreateTablet for tablet a9cfc7ea812a4139b28d6f1c793fe52f (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.107 [id=403570d6758d45bc9cc08db48cd768c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.758951 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cfc7ea812a4139b28d6f1c793fe52f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.763374 11411 tablet_bootstrap.cc:492] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.764712 11411 tablet_bootstrap.cc:654] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.767326 11411 tablet_bootstrap.cc:492] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.767510 11411 ts_tablet_manager.cc:1397] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:54.768134 11411 raft_consensus.cc:348] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.768343 11411 raft_consensus.cc:374] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.768426 11411 raft_consensus.cc:729] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.768683 11411 consensus_queue.cc:260] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.768853 11411 raft_consensus.cc:388] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.768951 11411 raft_consensus.cc:482] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.769053 11411 raft_consensus.cc:3037] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.770938 11411 raft_consensus.cc:504] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.771150 11411 leader_election.cc:302] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.771340 11411 leader_election.cc:288] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.771400 11413 raft_consensus.cc:2781] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.771597 11413 raft_consensus.cc:686] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.771925 11411 ts_tablet_manager.cc:1428] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:54.771870 11413 consensus_queue.cc:237] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.778061 11112 catalog_manager.cc:5495] T a9cfc7ea812a4139b28d6f1c793fe52f P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.792019 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:54.802733 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.009s sys 0.000s Metrics: {"bytes_written":6496,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":1380,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":580,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:54.803665 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 629 bytes on disk
I20260809 06:39:54.804193 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.804877 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.246157
I20260809 06:39:54.815310 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45176:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.108"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.828265 11297 tablet_service.cc:1467] Processing CreateTablet for tablet f2cfb3a140384e13bf6d4ab80096e9fe (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.108 [id=29def902608442bbb6cc091017b966a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.828730 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2cfb3a140384e13bf6d4ab80096e9fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.833999 11411 tablet_bootstrap.cc:492] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.834897 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.029s sys 0.000s Metrics: {"bytes_written":52753,"cfile_cache_hit":12,"cfile_cache_hit_bytes":40909,"cfile_cache_miss":28,"cfile_cache_miss_bytes":60253,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":783,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1309,"drs_written":1,"lbm_read_time_us":1189,"lbm_reads_lt_1ms":56,"lbm_write_time_us":969,"lbm_writes_lt_1ms":28,"mutex_wait_us":38,"num_input_rowsets":2,"peak_mem_usage":29056,"rows_written":217,"spinlock_wait_cycles":9984}
I20260809 06:39:54.835903 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000009
I20260809 06:39:54.836000 11411 tablet_bootstrap.cc:654] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.839282 11411 tablet_bootstrap.cc:492] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.839434 11411 ts_tablet_manager.cc:1397] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 06:39:54.840166 11411 raft_consensus.cc:348] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.840359 11411 raft_consensus.cc:374] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.840466 11411 raft_consensus.cc:729] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.840703 11411 consensus_queue.cc:260] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.840966 11411 raft_consensus.cc:388] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.841053 11411 raft_consensus.cc:482] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.841178 11411 raft_consensus.cc:3037] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.843214 11411 raft_consensus.cc:504] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.843482 11411 leader_election.cc:302] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.843703 11411 leader_election.cc:288] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.843763 11413 raft_consensus.cc:2781] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.843935 11413 raft_consensus.cc:686] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.844240 11411 ts_tablet_manager.cc:1428] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:54.844180 11413 consensus_queue.cc:237] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.847416 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.002s sys 0.007s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":191,"dirs.run_wall_time_us":1066,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":984,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:54.848325 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9493 bytes on disk
I20260809 06:39:54.850456 11112 catalog_manager.cc:5495] T f2cfb3a140384e13bf6d4ab80096e9fe P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.851954 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":154,"lbm_reads_lt_1ms":8}
I20260809 06:39:54.852689 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.222891
I20260809 06:39:54.854043 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:54.872920 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.009s sys 0.002s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":1159,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1206,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:54.873720 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.250959
I20260809 06:39:54.900638 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.043s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":41252,"cfile_cache_miss":28,"cfile_cache_miss_bytes":60502,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":625,"dirs.run_cpu_time_us":199,"dirs.run_wall_time_us":1232,"drs_written":1,"lbm_read_time_us":1129,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1067,"lbm_writes_lt_1ms":32,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":29139,"reinsert_count":0,"rows_written":219,"update_count":1}
I20260809 06:39:54.901925 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9324 bytes on disk
I20260809 06:39:54.902546 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.903007 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45184:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.109"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.908452 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.025s sys 0.008s Metrics: {"bytes_written":53181,"cfile_cache_hit":14,"cfile_cache_hit_bytes":41267,"cfile_cache_miss":34,"cfile_cache_miss_bytes":61497,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":980,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1177,"drs_written":1,"lbm_read_time_us":1147,"lbm_reads_lt_1ms":74,"lbm_write_time_us":862,"lbm_writes_lt_1ms":28,"mutex_wait_us":43,"num_input_rowsets":2,"peak_mem_usage":29412,"rows_written":219,"spinlock_wait_cycles":5888}
I20260809 06:39:54.909650 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260809 06:39:54.916663 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b91d6c05e5b642dfa3dac8d1e07efa6d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.109 [id=1d954fa0a83a478db8b43e4aaef5416e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.917268 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b91d6c05e5b642dfa3dac8d1e07efa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:54.922588 11411 tablet_bootstrap.cc:492] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:54.924085 11411 tablet_bootstrap.cc:654] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:54.927268 11411 tablet_bootstrap.cc:492] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:54.927474 11411 ts_tablet_manager.cc:1397] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:54.927800 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.009s sys 0.002s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":63,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1058,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1239,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:54.928261 11411 raft_consensus.cc:348] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.928514 11411 raft_consensus.cc:374] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:54.928544 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.537705
I20260809 06:39:54.928622 11411 raft_consensus.cc:729] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:54.928946 11411 consensus_queue.cc:260] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.929229 11411 raft_consensus.cc:388] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:54.929384 11411 raft_consensus.cc:482] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:54.929524 11411 raft_consensus.cc:3037] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:54.932204 11411 raft_consensus.cc:504] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.932475 11411 leader_election.cc:302] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:54.932721 11411 leader_election.cc:288] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:54.932806 11413 raft_consensus.cc:2781] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:54.932976 11413 raft_consensus.cc:686] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:54.933269 11411 ts_tablet_manager.cc:1428] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 06:39:54.933239 11413 consensus_queue.cc:237] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:54.938968 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":459,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":75,"lbm_read_time_us":500,"lbm_reads_lt_1ms":25,"lbm_write_time_us":288,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260809 06:39:54.939536 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9502 bytes on disk
I20260809 06:39:54.940259 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":163,"lbm_reads_lt_1ms":8}
I20260809 06:39:54.941128 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.220827
I20260809 06:39:54.945199 11112 catalog_manager.cc:5495] T b91d6c05e5b642dfa3dac8d1e07efa6d P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:54.976351 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.030s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":41649,"cfile_cache_miss":31,"cfile_cache_miss_bytes":61083,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1102,"dirs.run_cpu_time_us":219,"dirs.run_wall_time_us":1257,"drs_written":1,"lbm_read_time_us":1490,"lbm_reads_lt_1ms":63,"lbm_write_time_us":1343,"lbm_writes_lt_1ms":32,"mutex_wait_us":99,"num_input_rowsets":2,"peak_mem_usage":29410,"reinsert_count":0,"rows_written":221,"update_count":1}
I20260809 06:39:54.977682 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9403 bytes on disk
I20260809 06:39:54.978379 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260809 06:39:54.984917 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45188:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:54.996794 11297 tablet_service.cc:1467] Processing CreateTablet for tablet ce8d24ecbed241ecb39999c914197020 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.110 [id=25ffbb0bafc44b91a351c9bab7d75b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:54.997267 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8d24ecbed241ecb39999c914197020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.001732 11411 tablet_bootstrap.cc:492] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.003113 11411 tablet_bootstrap.cc:654] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.005784 11411 tablet_bootstrap.cc:492] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.005893 11411 ts_tablet_manager.cc:1397] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 06:39:55.006453 11411 raft_consensus.cc:348] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.006628 11411 raft_consensus.cc:374] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.006695 11411 raft_consensus.cc:729] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.006924 11411 consensus_queue.cc:260] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.007109 11411 raft_consensus.cc:388] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.007196 11411 raft_consensus.cc:482] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.007375 11411 raft_consensus.cc:3037] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.009235 11411 raft_consensus.cc:504] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.009559 11411 leader_election.cc:302] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.009742 11411 leader_election.cc:288] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.009838 11413 raft_consensus.cc:2781] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.010071 11413 raft_consensus.cc:686] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.010300 11413 consensus_queue.cc:237] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.010527 11411 ts_tablet_manager.cc:1428] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260809 06:39:55.017138 11112 catalog_manager.cc:5495] T ce8d24ecbed241ecb39999c914197020 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.052713 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45202:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.111"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.064520 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d9ac9f5af5b049e188480311dfffc6e0 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.111 [id=db8a29f628aa44228229fa1bd91431d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.065006 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ac9f5af5b049e188480311dfffc6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.069453 11411 tablet_bootstrap.cc:492] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.070942 11411 tablet_bootstrap.cc:654] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.073684 11411 tablet_bootstrap.cc:492] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.073853 11411 ts_tablet_manager.cc:1397] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:55.074469 11411 raft_consensus.cc:348] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.074726 11411 raft_consensus.cc:374] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.074828 11411 raft_consensus.cc:729] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.075098 11411 consensus_queue.cc:260] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.075292 11411 raft_consensus.cc:388] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.075392 11411 raft_consensus.cc:482] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.075493 11411 raft_consensus.cc:3037] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.077878 11411 raft_consensus.cc:504] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.078105 11411 leader_election.cc:302] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.078328 11411 leader_election.cc:288] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.078452 11413 raft_consensus.cc:2781] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.078651 11413 raft_consensus.cc:686] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.078833 11411 ts_tablet_manager.cc:1428] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:39:55.078876 11413 consensus_queue.cc:237] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.085610 11112 catalog_manager.cc:5495] T d9ac9f5af5b049e188480311dfffc6e0 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.123404 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45214:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.112"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.134867 11297 tablet_service.cc:1467] Processing CreateTablet for tablet fd20e3fb0e7e4b52abf487cb53169870 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.112 [id=4b314217256b406db1108dd576e563bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.135336 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd20e3fb0e7e4b52abf487cb53169870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.139689 11411 tablet_bootstrap.cc:492] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.141198 11411 tablet_bootstrap.cc:654] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.144208 11411 tablet_bootstrap.cc:492] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.144377 11411 ts_tablet_manager.cc:1397] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 06:39:55.145043 11411 raft_consensus.cc:348] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.145227 11411 raft_consensus.cc:374] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.145373 11411 raft_consensus.cc:729] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.145612 11411 consensus_queue.cc:260] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.145810 11411 raft_consensus.cc:388] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.145902 11411 raft_consensus.cc:482] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.146039 11411 raft_consensus.cc:3037] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.148029 11411 raft_consensus.cc:504] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.148241 11411 leader_election.cc:302] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.148415 11411 leader_election.cc:288] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.148483 11413 raft_consensus.cc:2781] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.148661 11413 raft_consensus.cc:686] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.148917 11413 consensus_queue.cc:237] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.149291 11411 ts_tablet_manager.cc:1428] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:55.154575 11112 catalog_manager.cc:5495] T fd20e3fb0e7e4b52abf487cb53169870 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.155862 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000086
I20260809 06:39:55.168584 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":333,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":65,"dirs.run_cpu_time_us":257,"dirs.run_wall_time_us":1423,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1158,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:55.169306 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.249012
I20260809 06:39:55.192194 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45216:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.113"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.203217 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.026s sys 0.006s Metrics: {"bytes_written":54939,"cfile_cache_hit":14,"cfile_cache_hit_bytes":42743,"cfile_cache_miss":34,"cfile_cache_miss_bytes":63605,"cfile_init":10,"delta_iterators_relevant":8,"dirs.queue_time_us":781,"dirs.run_cpu_time_us":1223,"dirs.run_wall_time_us":8374,"drs_written":1,"lbm_read_time_us":1179,"lbm_reads_lt_1ms":74,"lbm_write_time_us":1029,"lbm_writes_lt_1ms":28,"mutex_wait_us":131,"num_input_rowsets":2,"peak_mem_usage":30427,"rows_written":227,"spinlock_wait_cycles":5504,"thread_start_us":6409,"threads_started":1}
I20260809 06:39:55.204435 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9661 bytes on disk
I20260809 06:39:55.205114 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.205657 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 7aeef36a302c4da4ab30647e4ded7bdc (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.113 [id=af5cc2c207a8421bb96eaf7474bef668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.206152 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aeef36a302c4da4ab30647e4ded7bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.206214 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000010
I20260809 06:39:55.210886 11411 tablet_bootstrap.cc:492] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.212652 11411 tablet_bootstrap.cc:654] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.216616 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.000s sys 0.007s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":477,"dirs.run_cpu_time_us":194,"dirs.run_wall_time_us":1158,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":771,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:55.217262 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.164062
I20260809 06:39:55.229137 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000084
I20260809 06:39:55.233968 11411 tablet_bootstrap.cc:492] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.234180 11411 ts_tablet_manager.cc:1397] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260809 06:39:55.235106 11411 raft_consensus.cc:348] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.235359 11411 raft_consensus.cc:374] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.235463 11411 raft_consensus.cc:729] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.235741 11411 consensus_queue.cc:260] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.235980 11411 raft_consensus.cc:388] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.236093 11411 raft_consensus.cc:482] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.236364 11411 raft_consensus.cc:3037] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.238895 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.004s Metrics: {"bytes_written":6459,"cfile_init":1,"dirs.queue_time_us":76,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1310,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":725,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:55.239635 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.242395
I20260809 06:39:55.239743 11411 raft_consensus.cc:504] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.240037 11411 leader_election.cc:302] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.240334 11411 leader_election.cc:288] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.240417 11413 raft_consensus.cc:2781] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.240602 11413 raft_consensus.cc:686] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.240851 11411 ts_tablet_manager.cc:1428] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260809 06:39:55.240918 11413 consensus_queue.cc:237] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.247979 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.026s sys 0.004s Metrics: {"bytes_written":55348,"cfile_cache_hit":12,"cfile_cache_hit_bytes":43092,"cfile_cache_miss":28,"cfile_cache_miss_bytes":63170,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1044,"dirs.run_cpu_time_us":148,"dirs.run_wall_time_us":1106,"drs_written":1,"lbm_read_time_us":974,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1035,"lbm_writes_lt_1ms":28,"mutex_wait_us":27,"num_input_rowsets":2,"peak_mem_usage":30409,"rows_written":229,"spinlock_wait_cycles":5376}
I20260809 06:39:55.249184 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9736 bytes on disk
I20260809 06:39:55.249816 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.251739 11112 catalog_manager.cc:5495] T 7aeef36a302c4da4ab30647e4ded7bdc P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.274930 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.029s sys 0.004s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":43083,"cfile_cache_miss":31,"cfile_cache_miss_bytes":63719,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":772,"dirs.run_cpu_time_us":235,"dirs.run_wall_time_us":1282,"drs_written":1,"lbm_read_time_us":1419,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1190,"lbm_writes_lt_1ms":32,"mutex_wait_us":53,"num_input_rowsets":2,"peak_mem_usage":30458,"reinsert_count":0,"rows_written":229,"spinlock_wait_cycles":10112,"update_count":1}
I20260809 06:39:55.275827 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9736 bytes on disk
I20260809 06:39:55.276298 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.292806 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45222:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.114"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.304420 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 88f146429b9e415cb929405c65ad618e (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.114 [id=4039f5c2026f4c1b91690e1d865158a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.304970 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f146429b9e415cb929405c65ad618e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.309208 11411 tablet_bootstrap.cc:492] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.310745 11411 tablet_bootstrap.cc:654] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.313617 11411 tablet_bootstrap.cc:492] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.313774 11411 ts_tablet_manager.cc:1397] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 06:39:55.314396 11411 raft_consensus.cc:348] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.314572 11411 raft_consensus.cc:374] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.314713 11411 raft_consensus.cc:729] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.314934 11411 consensus_queue.cc:260] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.315146 11411 raft_consensus.cc:388] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.315243 11411 raft_consensus.cc:482] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.315375 11411 raft_consensus.cc:3037] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.317284 11411 raft_consensus.cc:504] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.317569 11411 leader_election.cc:302] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.317775 11411 leader_election.cc:288] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.317857 11413 raft_consensus.cc:2781] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.318053 11413 raft_consensus.cc:686] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.318286 11411 ts_tablet_manager.cc:1428] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:55.318320 11413 consensus_queue.cc:237] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.325155 11113 catalog_manager.cc:5495] T 88f146429b9e415cb929405c65ad618e P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.357312 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45236:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.368147 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b346d7e797344e8a81fa9edc131df370 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.115 [id=5fcd09c3f55b42989efef8d4ffdf9417]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.368605 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b346d7e797344e8a81fa9edc131df370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.372733 11411 tablet_bootstrap.cc:492] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.374123 11411 tablet_bootstrap.cc:654] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.376657 11411 tablet_bootstrap.cc:492] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.376844 11411 ts_tablet_manager.cc:1397] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:55.377502 11411 raft_consensus.cc:348] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.377717 11411 raft_consensus.cc:374] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.377811 11411 raft_consensus.cc:729] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.378067 11411 consensus_queue.cc:260] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.378239 11411 raft_consensus.cc:388] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.378355 11411 raft_consensus.cc:482] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.378468 11411 raft_consensus.cc:3037] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.380419 11411 raft_consensus.cc:504] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.380653 11411 leader_election.cc:302] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.380861 11411 leader_election.cc:288] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.380972 11413 raft_consensus.cc:2781] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.381177 11413 raft_consensus.cc:686] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.381454 11413 consensus_queue.cc:237] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.381695 11411 ts_tablet_manager.cc:1428] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:55.387235 11113 catalog_manager.cc:5495] T b346d7e797344e8a81fa9edc131df370 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.431062 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45244:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.116"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.444434 11297 tablet_service.cc:1467] Processing CreateTablet for tablet b262b74294fe42e89bea8999037a1720 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.116 [id=2f00a3806bad4ab3b81d114151dcebec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.444913 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b262b74294fe42e89bea8999037a1720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.451383 11411 tablet_bootstrap.cc:492] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.452924 11411 tablet_bootstrap.cc:654] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.456162 11411 tablet_bootstrap.cc:492] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.456315 11411 ts_tablet_manager.cc:1397] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:55.457101 11411 raft_consensus.cc:348] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.457275 11411 raft_consensus.cc:374] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.457409 11411 raft_consensus.cc:729] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.457650 11411 consensus_queue.cc:260] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.457864 11411 raft_consensus.cc:388] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.457978 11411 raft_consensus.cc:482] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.458122 11411 raft_consensus.cc:3037] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.460340 11411 raft_consensus.cc:504] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.460604 11411 leader_election.cc:302] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.460806 11411 leader_election.cc:288] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.460901 11413 raft_consensus.cc:2781] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.461069 11413 raft_consensus.cc:686] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.461362 11411 ts_tablet_manager.cc:1428] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:55.461319 11413 consensus_queue.cc:237] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.468372 11113 catalog_manager.cc:5495] T b262b74294fe42e89bea8999037a1720 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.503178 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:55.506829 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45250:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.117"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.516987 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.009s sys 0.000s Metrics: {"bytes_written":6065,"cfile_init":1,"dirs.queue_time_us":66,"dirs.run_cpu_time_us":211,"dirs.run_wall_time_us":1625,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":837,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6}
I20260809 06:39:55.517863 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.233272
I20260809 06:39:55.520766 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 06f2bed8c68947fdb2a0db9eee3c9f57 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.117 [id=0e1674b8d6e84b19ac45198c21684311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.521293 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f2bed8c68947fdb2a0db9eee3c9f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.526166 11411 tablet_bootstrap.cc:492] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.527259 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:55.527829 11411 tablet_bootstrap.cc:654] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.531061 11411 tablet_bootstrap.cc:492] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.531241 11411 ts_tablet_manager.cc:1397] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:55.532127 11411 raft_consensus.cc:348] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.532374 11411 raft_consensus.cc:374] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.532476 11411 raft_consensus.cc:729] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.532749 11411 consensus_queue.cc:260] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.532953 11411 raft_consensus.cc:388] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.533092 11411 raft_consensus.cc:482] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.533208 11411 raft_consensus.cc:3037] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.535943 11411 raft_consensus.cc:504] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.536216 11411 leader_election.cc:302] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.536458 11413 raft_consensus.cc:2781] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.536597 11413 raft_consensus.cc:686] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.536846 11413 consensus_queue.cc:237] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.539804 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.012s sys 0.000s Metrics: {"bytes_written":6458,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":1126,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":933,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:55.540705 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.239864
I20260809 06:39:55.544270 11411 leader_election.cc:288] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.545071 11411 ts_tablet_manager.cc:1428] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260809 06:39:55.546262 11113 catalog_manager.cc:5495] T 06f2bed8c68947fdb2a0db9eee3c9f57 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.556460 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.024s sys 0.008s Metrics: {"bytes_written":56685,"cfile_cache_hit":12,"cfile_cache_hit_bytes":44200,"cfile_cache_miss":28,"cfile_cache_miss_bytes":65002,"cfile_init":8,"delta_iterators_relevant":4,"dirs.queue_time_us":1074,"dirs.run_cpu_time_us":670,"dirs.run_wall_time_us":3963,"drs_written":1,"lbm_read_time_us":1152,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1197,"lbm_writes_lt_1ms":28,"mutex_wait_us":780,"num_input_rowsets":2,"peak_mem_usage":31169,"rows_written":235}
I20260809 06:39:55.558101 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10009 bytes on disk
I20260809 06:39:55.558864 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.559674 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260809 06:39:55.571776 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":329,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":58,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1170,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":720,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"spinlock_wait_cycles":28416,"update_count":1}
I20260809 06:39:55.572759 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.455679
I20260809 06:39:55.580137 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.029s sys 0.003s Metrics: {"bytes_written":386,"cfile_cache_hit":13,"cfile_cache_hit_bytes":44559,"cfile_cache_miss":31,"cfile_cache_miss_bytes":65895,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":708,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":979,"drs_written":1,"lbm_read_time_us":1723,"lbm_reads_lt_1ms":67,"lbm_write_time_us":990,"lbm_writes_lt_1ms":32,"mutex_wait_us":45,"num_input_rowsets":2,"peak_mem_usage":31474,"reinsert_count":0,"rows_written":237,"spinlock_wait_cycles":2304,"update_count":2}
I20260809 06:39:55.580935 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10077 bytes on disk
I20260809 06:39:55.581493 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.581945 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":572,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":134,"lbm_read_time_us":258,"lbm_reads_lt_1ms":21,"lbm_write_time_us":232,"lbm_writes_lt_1ms":9,"mutex_wait_us":18,"reinsert_count":0,"spinlock_wait_cycles":1920,"update_count":1}
I20260809 06:39:55.582533 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.195487
I20260809 06:39:55.610476 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45266:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.118"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.618523 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.027s sys 0.007s Metrics: {"bytes_written":57132,"cfile_cache_hit":13,"cfile_cache_hit_bytes":44636,"cfile_cache_miss":31,"cfile_cache_miss_bytes":65812,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":721,"dirs.run_cpu_time_us":197,"dirs.run_wall_time_us":1148,"drs_written":1,"lbm_read_time_us":909,"lbm_reads_lt_1ms":67,"lbm_write_time_us":1133,"lbm_writes_lt_1ms":28,"mutex_wait_us":71,"num_input_rowsets":2,"peak_mem_usage":31610,"rows_written":237,"spinlock_wait_cycles":14976}
I20260809 06:39:55.619493 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000013
I20260809 06:39:55.623023 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 24a2bcb3f62f4e3198d7c2e1b58b5990 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.118 [id=bacf600e098b48b1b819ef1b31d3a94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.623507 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a2bcb3f62f4e3198d7c2e1b58b5990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.628113 11411 tablet_bootstrap.cc:492] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.629807 11411 tablet_bootstrap.cc:654] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.631877 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.004s sys 0.004s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":36,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":948,"drs_written":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1515,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:55.632679 11411 tablet_bootstrap.cc:492] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.632718 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.537705
I20260809 06:39:55.632841 11411 ts_tablet_manager.cc:1397] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:55.633590 11411 raft_consensus.cc:348] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.633818 11411 raft_consensus.cc:374] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.633937 11411 raft_consensus.cc:729] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.634204 11411 consensus_queue.cc:260] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.634433 11411 raft_consensus.cc:388] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.634532 11411 raft_consensus.cc:482] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.634703 11411 raft_consensus.cc:3037] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.636488 11411 raft_consensus.cc:504] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.636750 11411 leader_election.cc:302] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.636991 11411 leader_election.cc:288] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.637065 11413 raft_consensus.cc:2781] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.637226 11413 raft_consensus.cc:686] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.637487 11413 consensus_queue.cc:237] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.638234 11411 ts_tablet_manager.cc:1428] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 06:39:55.640597 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.001s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":459,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":143,"lbm_read_time_us":420,"lbm_reads_lt_1ms":25,"lbm_write_time_us":281,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260809 06:39:55.641206 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.163691
I20260809 06:39:55.651288 11113 catalog_manager.cc:5495] T 24a2bcb3f62f4e3198d7c2e1b58b5990 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.670303 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.028s sys 0.000s Metrics: {"bytes_written":330,"cfile_cache_hit":13,"cfile_cache_hit_bytes":44975,"cfile_cache_miss":31,"cfile_cache_miss_bytes":65999,"cfile_init":10,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":960,"dirs.run_cpu_time_us":197,"dirs.run_wall_time_us":1090,"drs_written":1,"lbm_read_time_us":1153,"lbm_reads_lt_1ms":71,"lbm_write_time_us":1158,"lbm_writes_lt_1ms":32,"mutex_wait_us":113,"num_input_rowsets":2,"peak_mem_usage":31904,"reinsert_count":0,"rows_written":239,"update_count":1}
I20260809 06:39:55.671152 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260809 06:39:55.674160 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.002s sys 0.000s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":97,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:55.674578 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10168 bytes on disk
I20260809 06:39:55.675083 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.675567 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:55.680222 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":461,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":64,"lbm_read_time_us":189,"lbm_reads_lt_1ms":8,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 06:39:55.700227 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45280:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.119"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.711275 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdbf9c6c2f6405c996bf2dcbdbcb421 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.119 [id=d519e4e3e61249c8b068ebfc7e7bcf79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.711762 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdbf9c6c2f6405c996bf2dcbdbcb421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.716825 11411 tablet_bootstrap.cc:492] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.718333 11411 tablet_bootstrap.cc:654] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.720942 11411 tablet_bootstrap.cc:492] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.721117 11411 ts_tablet_manager.cc:1397] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:39:55.721808 11411 raft_consensus.cc:348] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.722009 11411 raft_consensus.cc:374] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.722101 11411 raft_consensus.cc:729] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.722328 11411 consensus_queue.cc:260] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.722604 11411 raft_consensus.cc:388] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.722712 11411 raft_consensus.cc:482] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.722802 11411 raft_consensus.cc:3037] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.724750 11411 raft_consensus.cc:504] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.724977 11411 leader_election.cc:302] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.725163 11411 leader_election.cc:288] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.725279 11413 raft_consensus.cc:2781] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.725517 11413 raft_consensus.cc:686] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.725620 11411 ts_tablet_manager.cc:1428] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260809 06:39:55.725759 11413 consensus_queue.cc:237] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.732146 11113 catalog_manager.cc:5495] T 5cdbf9c6c2f6405c996bf2dcbdbcb421 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.769886 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45294:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.120"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.781028 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 150f9d75e9c64047a3f717c3bab60330 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.120 [id=9d4fbef62da64cb1bded35fd10a844ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.781632 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150f9d75e9c64047a3f717c3bab60330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.786249 11411 tablet_bootstrap.cc:492] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.787619 11411 tablet_bootstrap.cc:654] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.790980 11411 tablet_bootstrap.cc:492] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.791101 11411 ts_tablet_manager.cc:1397] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:55.791772 11411 raft_consensus.cc:348] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.791935 11411 raft_consensus.cc:374] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.792042 11411 raft_consensus.cc:729] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.792320 11411 consensus_queue.cc:260] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.792536 11411 raft_consensus.cc:388] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.792611 11411 raft_consensus.cc:482] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.792690 11411 raft_consensus.cc:3037] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.794495 11411 raft_consensus.cc:504] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.794751 11411 leader_election.cc:302] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.794926 11411 leader_election.cc:288] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.794999 11413 raft_consensus.cc:2781] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.795173 11413 raft_consensus.cc:686] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.795526 11411 ts_tablet_manager.cc:1428] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:55.795409 11413 consensus_queue.cc:237] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.802425 11112 catalog_manager.cc:5495] T 150f9d75e9c64047a3f717c3bab60330 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.832523 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:55.835637 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45304:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.121"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.845583 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.003s Metrics: {"bytes_written":6066,"cfile_init":1,"dirs.queue_time_us":62,"dirs.run_cpu_time_us":227,"dirs.run_wall_time_us":1214,"drs_written":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4,"lbm_write_time_us":866,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":6,"spinlock_wait_cycles":38016}
I20260809 06:39:55.846478 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.200602
I20260809 06:39:55.849135 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d61fe1f5135742c19a2ca745e904ca11 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.121 [id=d887d063cd25484b9e4df45905aa4785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.849674 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61fe1f5135742c19a2ca745e904ca11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.855779 11411 tablet_bootstrap.cc:492] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.857424 11411 tablet_bootstrap.cc:654] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.860325 11411 tablet_bootstrap.cc:492] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.860486 11411 ts_tablet_manager.cc:1397] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:55.861104 11411 raft_consensus.cc:348] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.861320 11411 raft_consensus.cc:374] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.861449 11411 raft_consensus.cc:729] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.861735 11411 consensus_queue.cc:260] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.861977 11413 raft_consensus.cc:482] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:39:55.862100 11413 raft_consensus.cc:504] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.861951 11411 raft_consensus.cc:388] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.862355 11413 leader_election.cc:302] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.862433 11411 raft_consensus.cc:482] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.862552 11411 raft_consensus.cc:3037] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.862576 11413 leader_election.cc:288] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 06:39:55.865204 11411 raft_consensus.cc:504] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.865510 11411 leader_election.cc:302] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.865598 11415 raft_consensus.cc:2741] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 06:39:55.865731 11411 leader_election.cc:288] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.865818 11413 raft_consensus.cc:2781] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.866001 11413 raft_consensus.cc:686] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.866218 11413 consensus_queue.cc:237] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.866726 11411 ts_tablet_manager.cc:1428] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 06:39:55.872908 11112 catalog_manager.cc:5495] T d61fe1f5135742c19a2ca745e904ca11 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.884984 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.028s sys 0.009s Metrics: {"bytes_written":58428,"cfile_cache_hit":13,"cfile_cache_hit_bytes":45710,"cfile_cache_miss":31,"cfile_cache_miss_bytes":67766,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":736,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":1049,"drs_written":1,"lbm_read_time_us":1300,"lbm_reads_lt_1ms":67,"lbm_write_time_us":945,"lbm_writes_lt_1ms":28,"mutex_wait_us":46,"num_input_rowsets":2,"peak_mem_usage":32817,"rows_written":243,"spinlock_wait_cycles":3968}
I20260809 06:39:55.885951 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10372 bytes on disk
I20260809 06:39:55.886456 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.887131 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260809 06:39:55.896571 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"bytes_written":332,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":47,"dirs.run_cpu_time_us":166,"dirs.run_wall_time_us":951,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":854,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":2,"update_count":1}
I20260809 06:39:55.897526 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 357 bytes on disk
I20260809 06:39:55.898617 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.899154 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.439153
I20260809 06:39:55.905412 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":609,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":153,"lbm_read_time_us":255,"lbm_reads_lt_1ms":21,"lbm_write_time_us":376,"lbm_writes_lt_1ms":9,"reinsert_count":0,"update_count":1}
I20260809 06:39:55.905884 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.195632
I20260809 06:39:55.924117 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45312:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.122"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:55.931106 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260809 06:39:55.935036 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.027s sys 0.000s Metrics: {"bytes_written":58877,"cfile_cache_hit":13,"cfile_cache_hit_bytes":46116,"cfile_cache_miss":31,"cfile_cache_miss_bytes":68260,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1435,"dirs.run_cpu_time_us":809,"dirs.run_wall_time_us":4740,"drs_written":1,"lbm_read_time_us":958,"lbm_reads_lt_1ms":63,"lbm_write_time_us":998,"lbm_writes_lt_1ms":29,"mutex_wait_us":40,"num_input_rowsets":2,"peak_mem_usage":33241,"rows_written":245,"spinlock_wait_cycles":8192}
I20260809 06:39:55.935992 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10671 bytes on disk
I20260809 06:39:55.936614 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.937592 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:55.938278 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3cae1998a8e94e188f6d8124e3909700 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.122 [id=52d615c3f934458dbff7ff7945a9a52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:55.938797 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cae1998a8e94e188f6d8124e3909700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:55.944296 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.009s sys 0.000s Metrics: {"bytes_written":328,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":70,"dirs.run_cpu_time_us":161,"dirs.run_wall_time_us":3317,"drs_written":1,"lbm_read_time_us":114,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1320,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":8,"update_count":1}
I20260809 06:39:55.945218 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 566 bytes on disk
I20260809 06:39:55.945583 11411 tablet_bootstrap.cc:492] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:55.946910 11411 tablet_bootstrap.cc:654] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:55.947871 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.001s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":69,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":942,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":937,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:55.948258 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":124,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.948596 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:55.948928 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.240411
I20260809 06:39:55.949543 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.949913 11411 tablet_bootstrap.cc:492] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:55.950069 11411 ts_tablet_manager.cc:1397] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:55.950134 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.187995
I20260809 06:39:55.950937 11411 raft_consensus.cc:348] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.951157 11411 raft_consensus.cc:374] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:55.951270 11411 raft_consensus.cc:729] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:55.951537 11411 consensus_queue.cc:260] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.951783 11411 raft_consensus.cc:388] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:55.951876 11411 raft_consensus.cc:482] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:55.952010 11411 raft_consensus.cc:3037] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:55.954946 11411 raft_consensus.cc:504] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.955233 11411 leader_election.cc:302] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:55.955453 11411 leader_election.cc:288] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:55.955535 11413 raft_consensus.cc:2781] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:55.955750 11413 raft_consensus.cc:686] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:55.955971 11411 ts_tablet_manager.cc:1428] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260809 06:39:55.956000 11413 consensus_queue.cc:237] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:55.963909 11112 catalog_manager.cc:5495] T 3cae1998a8e94e188f6d8124e3909700 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:55.983534 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.028s sys 0.003s Metrics: {"bytes_written":333,"cfile_cache_hit":12,"cfile_cache_hit_bytes":46437,"cfile_cache_miss":30,"cfile_cache_miss_bytes":68523,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":623,"dirs.run_cpu_time_us":157,"dirs.run_wall_time_us":923,"drs_written":1,"lbm_read_time_us":1190,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1402,"lbm_writes_lt_1ms":33,"mutex_wait_us":135,"num_input_rowsets":2,"peak_mem_usage":33229,"reinsert_count":0,"rows_written":247,"spinlock_wait_cycles":26368,"update_count":1}
I20260809 06:39:55.983637 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.024s sys 0.008s Metrics: {"bytes_written":333,"cfile_cache_hit":14,"cfile_cache_hit_bytes":46321,"cfile_cache_miss":34,"cfile_cache_miss_bytes":68977,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":8,"dirs.queue_time_us":818,"dirs.run_cpu_time_us":738,"dirs.run_wall_time_us":3810,"drs_written":1,"lbm_read_time_us":1463,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1470,"lbm_writes_lt_1ms":33,"mutex_wait_us":435,"num_input_rowsets":2,"peak_mem_usage":33482,"reinsert_count":0,"rows_written":247,"spinlock_wait_cycles":2944,"update_count":1}
I20260809 06:39:55.984485 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10719 bytes on disk
I20260809 06:39:55.984519 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10719 bytes on disk
I20260809 06:39:55.984989 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260809 06:39:55.985042 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.011101 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45326:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.123"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.022631 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 30a743accafb418f8bda6f6bf4438523 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.123 [id=8aca87acd8624422bcfa4d5653dbfc0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.023118 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30a743accafb418f8bda6f6bf4438523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.027424 11411 tablet_bootstrap.cc:492] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.029006 11411 tablet_bootstrap.cc:654] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.032423 11411 tablet_bootstrap.cc:492] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.032599 11411 ts_tablet_manager.cc:1397] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 06:39:56.033281 11411 raft_consensus.cc:348] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.033540 11411 raft_consensus.cc:374] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.033630 11411 raft_consensus.cc:729] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.033869 11411 consensus_queue.cc:260] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.034039 11411 raft_consensus.cc:388] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.034119 11411 raft_consensus.cc:482] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.034194 11411 raft_consensus.cc:3037] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.036792 11411 raft_consensus.cc:504] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.037043 11411 leader_election.cc:302] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.037256 11411 leader_election.cc:288] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.037765 11413 raft_consensus.cc:2781] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.037945 11413 raft_consensus.cc:686] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.038086 11411 ts_tablet_manager.cc:1428] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:56.038234 11413 consensus_queue.cc:237] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.047313 11113 catalog_manager.cc:5495] T 30a743accafb418f8bda6f6bf4438523 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.079576 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45336:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.124"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.090528 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 82f5c30dfb9b45c3a3792809247782e4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.124 [id=6ebeb4974172439eb372c5b5e30b7dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.091058 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f5c30dfb9b45c3a3792809247782e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.095913 11411 tablet_bootstrap.cc:492] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.097487 11411 tablet_bootstrap.cc:654] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.099944 11411 tablet_bootstrap.cc:492] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.100107 11411 ts_tablet_manager.cc:1397] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:56.100761 11411 raft_consensus.cc:348] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.100978 11411 raft_consensus.cc:374] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.101109 11411 raft_consensus.cc:729] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.101434 11411 consensus_queue.cc:260] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.101683 11411 raft_consensus.cc:388] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.101778 11411 raft_consensus.cc:482] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.101866 11411 raft_consensus.cc:3037] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.103639 11411 raft_consensus.cc:504] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.103852 11411 leader_election.cc:302] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.104025 11411 leader_election.cc:288] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.104122 11413 raft_consensus.cc:2781] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.104317 11413 raft_consensus.cc:686] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.104548 11413 consensus_queue.cc:237] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.104679 11411 ts_tablet_manager.cc:1428] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260809 06:39:56.112689 11113 catalog_manager.cc:5495] T 82f5c30dfb9b45c3a3792809247782e4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.144477 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45344:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.125"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.156806 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 49227f8b298e4c388261d42b6a10739d (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.125 [id=9b8e8db6468549d59cb9eabac691887d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.157290 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49227f8b298e4c388261d42b6a10739d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.161859 11411 tablet_bootstrap.cc:492] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.163597 11411 tablet_bootstrap.cc:654] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.166297 11411 tablet_bootstrap.cc:492] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.166429 11411 ts_tablet_manager.cc:1397] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:56.167177 11411 raft_consensus.cc:348] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.167440 11411 raft_consensus.cc:374] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.167532 11411 raft_consensus.cc:729] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.167827 11411 consensus_queue.cc:260] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.167995 11411 raft_consensus.cc:388] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.168111 11411 raft_consensus.cc:482] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.168273 11411 raft_consensus.cc:3037] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.170157 11411 raft_consensus.cc:504] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.170390 11411 leader_election.cc:302] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.170604 11411 leader_election.cc:288] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.170675 11413 raft_consensus.cc:2781] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.170857 11413 raft_consensus.cc:686] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.171123 11413 consensus_queue.cc:237] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.172269 11411 ts_tablet_manager.cc:1428] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 06:39:56.177959 11113 catalog_manager.cc:5495] T 49227f8b298e4c388261d42b6a10739d P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.217384 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45358:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.126"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.228335 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5ea7b338054fe0a16c580455b38f3c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.126 [id=93ad24161547404c8675d6f977276cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.228716 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b5ea7b338054fe0a16c580455b38f3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.233458 11411 tablet_bootstrap.cc:492] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.234983 11411 tablet_bootstrap.cc:654] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.235906 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:56.236037 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000080
I20260809 06:39:56.238266 11411 tablet_bootstrap.cc:492] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.238413 11411 ts_tablet_manager.cc:1397] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 06:39:56.239153 11411 raft_consensus.cc:348] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.239333 11411 raft_consensus.cc:374] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.239475 11411 raft_consensus.cc:729] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.239755 11411 consensus_queue.cc:260] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.240020 11411 raft_consensus.cc:388] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.240149 11411 raft_consensus.cc:482] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.240275 11411 raft_consensus.cc:3037] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.243058 11411 raft_consensus.cc:504] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.243363 11411 leader_election.cc:302] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.243603 11411 leader_election.cc:288] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.243676 11413 raft_consensus.cc:2781] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.243863 11413 raft_consensus.cc:686] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.244095 11413 consensus_queue.cc:237] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.244634 11411 ts_tablet_manager.cc:1428] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 06:39:56.247560 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":6457,"cfile_init":1,"dirs.queue_time_us":80,"dirs.run_cpu_time_us":165,"dirs.run_wall_time_us":3918,"drs_written":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1047,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.248677 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 613 bytes on disk
I20260809 06:39:56.253481 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.254258 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.208553
I20260809 06:39:56.254225 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.009s sys 0.003s Metrics: {"bytes_written":6457,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1298,"drs_written":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1129,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.257234 11113 catalog_manager.cc:5495] T 3b5ea7b338054fe0a16c580455b38f3c P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.258015 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.208553
I20260809 06:39:56.300141 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.034s sys 0.005s Metrics: {"bytes_written":386,"cfile_cache_hit":13,"cfile_cache_hit_bytes":47895,"cfile_cache_miss":33,"cfile_cache_miss_bytes":71223,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":2464,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":899,"drs_written":1,"lbm_read_time_us":1753,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1651,"lbm_writes_lt_1ms":34,"num_input_rowsets":2,"peak_mem_usage":34061,"reinsert_count":0,"rows_written":255,"spinlock_wait_cycles":896,"update_count":2}
I20260809 06:39:56.300172 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.030s sys 0.011s Metrics: {"bytes_written":386,"cfile_cache_hit":13,"cfile_cache_hit_bytes":47895,"cfile_cache_miss":33,"cfile_cache_miss_bytes":71223,"cfile_init":8,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":874,"dirs.run_cpu_time_us":796,"dirs.run_wall_time_us":4368,"drs_written":1,"lbm_read_time_us":4636,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1579,"lbm_writes_lt_1ms":34,"mutex_wait_us":841,"num_input_rowsets":2,"peak_mem_usage":34061,"reinsert_count":0,"rows_written":255,"update_count":2}
I20260809 06:39:56.301103 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11096 bytes on disk
I20260809 06:39:56.301298 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11096 bytes on disk
I20260809 06:39:56.301844 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.302196 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":107,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.312211 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45374:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.127"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.324045 11297 tablet_service.cc:1467] Processing CreateTablet for tablet be9383a143c8497fa55d13e5d634cbac (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.127 [id=22417a7855fe4bb985626fa09204187e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.324538 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9383a143c8497fa55d13e5d634cbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.328909 11411 tablet_bootstrap.cc:492] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.330382 11411 tablet_bootstrap.cc:654] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.333158 11411 tablet_bootstrap.cc:492] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.333387 11411 ts_tablet_manager.cc:1397] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:56.334034 11411 raft_consensus.cc:348] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.334232 11411 raft_consensus.cc:374] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.334311 11411 raft_consensus.cc:729] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.334571 11411 consensus_queue.cc:260] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.334767 11411 raft_consensus.cc:388] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.334867 11411 raft_consensus.cc:482] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.334955 11411 raft_consensus.cc:3037] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.336612 11411 raft_consensus.cc:504] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.336843 11411 leader_election.cc:302] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.337054 11411 leader_election.cc:288] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.337184 11413 raft_consensus.cc:2781] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.337409 11413 raft_consensus.cc:686] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.337646 11411 ts_tablet_manager.cc:1428] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260809 06:39:56.337677 11413 consensus_queue.cc:237] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.345927 11113 catalog_manager.cc:5495] T be9383a143c8497fa55d13e5d634cbac P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.379444 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45382:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.128"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.390916 11297 tablet_service.cc:1467] Processing CreateTablet for tablet c2162d3f0f6246cbba507ee7d3825ab4 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.128 [id=0c865df9917146e99e557f9b550898f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.391383 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2162d3f0f6246cbba507ee7d3825ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.397732 11411 tablet_bootstrap.cc:492] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.399308 11411 tablet_bootstrap.cc:654] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.402249 11411 tablet_bootstrap.cc:492] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.402401 11411 ts_tablet_manager.cc:1397] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:56.403033 11411 raft_consensus.cc:348] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.403198 11411 raft_consensus.cc:374] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.403314 11411 raft_consensus.cc:729] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.403546 11411 consensus_queue.cc:260] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.403738 11411 raft_consensus.cc:388] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.403826 11411 raft_consensus.cc:482] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.403988 11411 raft_consensus.cc:3037] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.406220 11411 raft_consensus.cc:504] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.406513 11411 leader_election.cc:302] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.406736 11411 leader_election.cc:288] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.406821 11413 raft_consensus.cc:2781] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.406998 11413 raft_consensus.cc:686] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.407284 11411 ts_tablet_manager.cc:1428] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:56.407239 11413 consensus_queue.cc:237] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.414981 11113 catalog_manager.cc:5495] T c2162d3f0f6246cbba507ee7d3825ab4 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.458953 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45384:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.129"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.473552 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3be2cbee516e4ae0a97b271ec3912d37 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.129 [id=1b9d2d6f01b1463088de24a5a6db9538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.474032 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be2cbee516e4ae0a97b271ec3912d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.480291 11411 tablet_bootstrap.cc:492] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.481748 11411 tablet_bootstrap.cc:654] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.484678 11411 tablet_bootstrap.cc:492] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.484865 11411 ts_tablet_manager.cc:1397] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 06:39:56.485515 11411 raft_consensus.cc:348] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.485719 11411 raft_consensus.cc:374] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.485816 11411 raft_consensus.cc:729] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.486069 11411 consensus_queue.cc:260] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.486243 11411 raft_consensus.cc:388] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.486359 11411 raft_consensus.cc:482] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.486477 11411 raft_consensus.cc:3037] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.488312 11411 raft_consensus.cc:504] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.488566 11411 leader_election.cc:302] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.488855 11411 leader_election.cc:288] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.488974 11413 raft_consensus.cc:2781] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.489176 11413 raft_consensus.cc:686] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.489542 11413 consensus_queue.cc:237] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.489641 11411 ts_tablet_manager.cc:1428] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:56.496997 11113 catalog_manager.cc:5495] T 3be2cbee516e4ae0a97b271ec3912d37 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.532770 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45396:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.130"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.544354 11297 tablet_service.cc:1467] Processing CreateTablet for tablet d5cb70b4624243a0a296822a44b89097 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.130 [id=feaf55e8bd3d4abb90dc1d4d9f0ba037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.544898 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5cb70b4624243a0a296822a44b89097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.549774 11411 tablet_bootstrap.cc:492] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.551265 11411 tablet_bootstrap.cc:654] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.552922 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000085
I20260809 06:39:56.553117 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260809 06:39:56.554169 11411 tablet_bootstrap.cc:492] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.554342 11411 ts_tablet_manager.cc:1397] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:56.555375 11411 raft_consensus.cc:348] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.555719 11411 raft_consensus.cc:374] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.555841 11411 raft_consensus.cc:729] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.556119 11411 consensus_queue.cc:260] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.556347 11411 raft_consensus.cc:388] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.556439 11411 raft_consensus.cc:482] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.556564 11411 raft_consensus.cc:3037] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.559257 11411 raft_consensus.cc:504] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.559561 11411 leader_election.cc:302] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.559774 11411 leader_election.cc:288] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.559846 11413 raft_consensus.cc:2781] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.560017 11413 raft_consensus.cc:686] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.560232 11413 consensus_queue.cc:237] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.560760 11411 ts_tablet_manager.cc:1428] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 06:39:56.564685 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.009s sys 0.000s Metrics: {"bytes_written":6459,"cfile_init":1,"dirs.queue_time_us":71,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":4711,"drs_written":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1124,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.565608 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.237985
I20260809 06:39:56.568045 11113 catalog_manager.cc:5495] T d5cb70b4624243a0a296822a44b89097 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.577785 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.021s sys 0.000s Metrics: {"bytes_written":6459,"cfile_init":1,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":181,"dirs.run_wall_time_us":1094,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":877,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.578730 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 585 bytes on disk
I20260809 06:39:56.579849 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.580596 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.237985
I20260809 06:39:56.605379 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.036s sys 0.001s Metrics: {"bytes_written":333,"cfile_cache_hit":13,"cfile_cache_hit_bytes":49371,"cfile_cache_miss":33,"cfile_cache_miss_bytes":73503,"cfile_init":9,"delete_count":0,"delta_iterators_relevant":6,"dirs.queue_time_us":1324,"dirs.run_cpu_time_us":211,"dirs.run_wall_time_us":1114,"drs_written":1,"lbm_read_time_us":1526,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1325,"lbm_writes_lt_1ms":34,"mutex_wait_us":27,"num_input_rowsets":2,"peak_mem_usage":35845,"reinsert_count":0,"rows_written":263,"spinlock_wait_cycles":21120,"update_count":1}
I20260809 06:39:56.606699 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11447 bytes on disk
I20260809 06:39:56.607355 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.614998 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45400:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.615577 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.032s sys 0.002s Metrics: {"bytes_written":62830,"cfile_cache_hit":13,"cfile_cache_hit_bytes":49371,"cfile_cache_miss":33,"cfile_cache_miss_bytes":73503,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":1085,"dirs.run_cpu_time_us":124,"dirs.run_wall_time_us":1119,"drs_written":1,"lbm_read_time_us":1400,"lbm_reads_lt_1ms":65,"lbm_write_time_us":1134,"lbm_writes_lt_1ms":30,"mutex_wait_us":39,"num_input_rowsets":2,"peak_mem_usage":35845,"rows_written":263,"spinlock_wait_cycles":2816,"thread_start_us":284,"threads_started":1}
I20260809 06:39:56.616472 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11494 bytes on disk
I20260809 06:39:56.617012 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.627406 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1f4bea4bdeca49c29b2a52e27e99ff59 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.131 [id=3c88f365e7f9425c8dda7bdabb695195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.627828 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f4bea4bdeca49c29b2a52e27e99ff59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.632652 11411 tablet_bootstrap.cc:492] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.634368 11411 tablet_bootstrap.cc:654] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.636958 11411 tablet_bootstrap.cc:492] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.637133 11411 ts_tablet_manager.cc:1397] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:56.637981 11411 raft_consensus.cc:348] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.638216 11411 raft_consensus.cc:374] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.638298 11411 raft_consensus.cc:729] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.638553 11411 consensus_queue.cc:260] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.638724 11411 raft_consensus.cc:388] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.638847 11411 raft_consensus.cc:482] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.638960 11411 raft_consensus.cc:3037] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.640905 11411 raft_consensus.cc:504] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.641132 11411 leader_election.cc:302] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.641408 11411 leader_election.cc:288] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.641541 11413 raft_consensus.cc:2781] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.641742 11413 raft_consensus.cc:686] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.641887 11411 ts_tablet_manager.cc:1428] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260809 06:39:56.642094 11413 consensus_queue.cc:237] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.648204 11113 catalog_manager.cc:5495] T 1f4bea4bdeca49c29b2a52e27e99ff59 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.682464 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45416:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.693677 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 36271dd4f8924f76b69660714b6e79c7 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.132 [id=de54f5aebdcb4fcab7e1e92614a1e13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.694161 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36271dd4f8924f76b69660714b6e79c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.699026 11411 tablet_bootstrap.cc:492] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.700342 11411 tablet_bootstrap.cc:654] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.703398 11411 tablet_bootstrap.cc:492] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.703512 11411 ts_tablet_manager.cc:1397] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:56.704042 11411 raft_consensus.cc:348] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.704196 11411 raft_consensus.cc:374] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.704264 11411 raft_consensus.cc:729] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.704473 11411 consensus_queue.cc:260] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.704628 11411 raft_consensus.cc:388] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.704689 11411 raft_consensus.cc:482] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.704802 11411 raft_consensus.cc:3037] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.706683 11411 raft_consensus.cc:504] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.706887 11411 leader_election.cc:302] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.707064 11411 leader_election.cc:288] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.707124 11413 raft_consensus.cc:2781] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.707357 11413 raft_consensus.cc:686] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.707680 11411 ts_tablet_manager.cc:1428] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 06:39:56.707602 11413 consensus_queue.cc:237] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.713922 11113 catalog_manager.cc:5495] T 36271dd4f8924f76b69660714b6e79c7 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.750094 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45424:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.133"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.762370 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7e9a7e64b649cbac0dab8cbe26f928 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.133 [id=dcf3c31c2aaf4f25a5bb2abede6fa153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.762884 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7e9a7e64b649cbac0dab8cbe26f928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.767532 11411 tablet_bootstrap.cc:492] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.769119 11411 tablet_bootstrap.cc:654] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.772115 11411 tablet_bootstrap.cc:492] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.772307 11411 ts_tablet_manager.cc:1397] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:39:56.773059 11411 raft_consensus.cc:348] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.773247 11411 raft_consensus.cc:374] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.773362 11411 raft_consensus.cc:729] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.773617 11411 consensus_queue.cc:260] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.773799 11411 raft_consensus.cc:388] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.773921 11411 raft_consensus.cc:482] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.774066 11411 raft_consensus.cc:3037] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.775751 11411 raft_consensus.cc:504] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.775982 11411 leader_election.cc:302] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.776249 11411 leader_election.cc:288] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.776325 11413 raft_consensus.cc:2781] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.776549 11413 raft_consensus.cc:686] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.776829 11413 consensus_queue.cc:237] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.777076 11411 ts_tablet_manager.cc:1428] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 06:39:56.783216 11113 catalog_manager.cc:5495] T 3e7e9a7e64b649cbac0dab8cbe26f928 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.816776 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45428:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.134"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.828773 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2cfed33fe8473694a60201c4690011 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.134 [id=6689231b25784a948ef8cc7513c830fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.829203 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2cfed33fe8473694a60201c4690011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.833803 11411 tablet_bootstrap.cc:492] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.835377 11411 tablet_bootstrap.cc:654] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.837963 11411 tablet_bootstrap.cc:492] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.838124 11411 ts_tablet_manager.cc:1397] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:39:56.838811 11411 raft_consensus.cc:348] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.839000 11411 raft_consensus.cc:374] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.839118 11411 raft_consensus.cc:729] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.839336 11411 consensus_queue.cc:260] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.839571 11411 raft_consensus.cc:388] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.839668 11411 raft_consensus.cc:482] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.839793 11411 raft_consensus.cc:3037] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.841812 11411 raft_consensus.cc:504] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.842027 11411 leader_election.cc:302] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.842177 11411 leader_election.cc:288] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.842263 11413 raft_consensus.cc:2781] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.842525 11413 raft_consensus.cc:686] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.842698 11411 ts_tablet_manager.cc:1428] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:39:56.842785 11413 consensus_queue.cc:237] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.850000 11112 catalog_manager.cc:5495] T 1c2cfed33fe8473694a60201c4690011 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.858250 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000082
I20260809 06:39:56.868016 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000081
I20260809 06:39:56.869412 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.010s sys 0.000s Metrics: {"bytes_written":6493,"cfile_init":1,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1192,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":910,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.870268 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.206256
I20260809 06:39:56.880038 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":6493,"cfile_init":1,"dirs.queue_time_us":37,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":1063,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1050,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":8}
I20260809 06:39:56.881283 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 597 bytes on disk
I20260809 06:39:56.881955 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.882622 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.206428
I20260809 06:39:56.892021 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45432:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.135"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:56.906492 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 07f98340a7a64a94ae1495585478f9b3 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.135 [id=34227913b8fa408280c4d7b314027373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:56.906970 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f98340a7a64a94ae1495585478f9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:56.907469 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.036s sys 0.000s Metrics: {"bytes_written":64560,"cfile_cache_hit":13,"cfile_cache_hit_bytes":50884,"cfile_cache_miss":33,"cfile_cache_miss_bytes":75668,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":515,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":987,"drs_written":1,"lbm_read_time_us":1550,"lbm_reads_lt_1ms":69,"lbm_write_time_us":1044,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":36029,"rows_written":271}
I20260809 06:39:56.908247 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11820 bytes on disk
I20260809 06:39:56.909253 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.909854 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000011
I20260809 06:39:56.913470 11411 tablet_bootstrap.cc:492] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:56.915313 11411 tablet_bootstrap.cc:654] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:56.918740 11411 tablet_bootstrap.cc:492] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:56.918960 11411 ts_tablet_manager.cc:1397] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 06:39:56.919839 11411 raft_consensus.cc:348] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.920063 11411 raft_consensus.cc:374] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:56.920146 11411 raft_consensus.cc:729] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:56.920012 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":1757,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1071,"drs_written":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1042,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:56.920382 11411 consensus_queue.cc:260] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.920588 11411 raft_consensus.cc:388] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:56.920714 11411 raft_consensus.cc:482] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:56.920820 11411 raft_consensus.cc:3037] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:56.920881 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 235 bytes on disk
I20260809 06:39:56.921463 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":130,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.922076 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.186072
I20260809 06:39:56.922096 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.032s sys 0.000s Metrics: {"bytes_written":64560,"cfile_cache_hit":12,"cfile_cache_hit_bytes":50887,"cfile_cache_miss":30,"cfile_cache_miss_bytes":75369,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":417,"dirs.run_cpu_time_us":217,"dirs.run_wall_time_us":1102,"drs_written":1,"lbm_read_time_us":1141,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1214,"lbm_writes_lt_1ms":30,"mutex_wait_us":68,"num_input_rowsets":2,"peak_mem_usage":35981,"rows_written":271}
I20260809 06:39:56.922940 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000012
I20260809 06:39:56.923931 11411 raft_consensus.cc:504] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.924252 11411 leader_election.cc:302] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:56.924533 11413 raft_consensus.cc:2781] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:56.924690 11413 raft_consensus.cc:686] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:56.924932 11413 consensus_queue.cc:237] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:56.930812 11411 leader_election.cc:288] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:56.931545 11411 ts_tablet_manager.cc:1428] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260809 06:39:56.932200 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.000s Metrics: {"bytes_written":5157,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":146,"dirs.run_wall_time_us":963,"drs_written":1,"lbm_read_time_us":46,"lbm_reads_lt_1ms":4,"lbm_write_time_us":679,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 06:39:56.932826 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 12055 bytes on disk
I20260809 06:39:56.933689 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":182,"lbm_reads_lt_1ms":8}
I20260809 06:39:56.934350 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.186072
I20260809 06:39:56.944352 11112 catalog_manager.cc:5495] T 07f98340a7a64a94ae1495585478f9b3 P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:56.965327 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.039s sys 0.003s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":51230,"cfile_cache_miss":30,"cfile_cache_miss_bytes":75628,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":1183,"dirs.run_cpu_time_us":168,"dirs.run_wall_time_us":1242,"drs_written":1,"lbm_read_time_us":1633,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1477,"lbm_writes_lt_1ms":34,"mutex_wait_us":44,"num_input_rowsets":2,"peak_mem_usage":37022,"reinsert_count":0,"rows_written":273,"spinlock_wait_cycles":11264,"update_count":1}
I20260809 06:39:56.966296 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11919 bytes on disk
I20260809 06:39:56.966988 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":128,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.970031 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.028s sys 0.005s Metrics: {"bytes_written":330,"cfile_cache_hit":12,"cfile_cache_hit_bytes":51230,"cfile_cache_miss":30,"cfile_cache_miss_bytes":75628,"cfile_init":7,"delete_count":0,"delta_iterators_relevant":4,"dirs.queue_time_us":822,"dirs.run_cpu_time_us":290,"dirs.run_wall_time_us":1595,"drs_written":1,"lbm_read_time_us":1150,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1309,"lbm_writes_lt_1ms":34,"num_input_rowsets":2,"peak_mem_usage":37022,"reinsert_count":0,"rows_written":273,"update_count":1}
I20260809 06:39:56.970970 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11919 bytes on disk
I20260809 06:39:56.971510 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260809 06:39:56.972069 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000001
I20260809 06:39:56.975267 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.001s sys 0.000s Metrics: {"bytes_written":333,"delete_count":0,"lbm_write_time_us":134,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 06:39:56.975718 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 06:39:56.979848 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":461,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":59,"lbm_read_time_us":93,"lbm_reads_lt_1ms":8,"lbm_write_time_us":105,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 06:39:56.997851 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45440:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.136"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:57.009614 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 3e41021de9d84d32a3cf95972e93461c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.136 [id=2495074c7f864729bac03a2bd29aefb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:57.010133 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e41021de9d84d32a3cf95972e93461c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:57.014676 11411 tablet_bootstrap.cc:492] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:57.016384 11411 tablet_bootstrap.cc:654] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
I20260809 06:39:57.019884 11411 tablet_bootstrap.cc:492] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027: No bootstrap required, opened a new log
I20260809 06:39:57.020046 11411 ts_tablet_manager.cc:1397] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 06:39:57.021086 11411 raft_consensus.cc:348] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:57.021294 11411 raft_consensus.cc:374] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:39:57.021413 11411 raft_consensus.cc:729] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Initialized, Role: FOLLOWER
I20260809 06:39:57.021652 11411 consensus_queue.cc:260] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:57.021848 11411 raft_consensus.cc:388] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:39:57.021955 11411 raft_consensus.cc:482] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:39:57.022070 11411 raft_consensus.cc:3037] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:39:57.023928 11411 raft_consensus.cc:504] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:57.024175 11411 leader_election.cc:302] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4f2c9d8f334254a624a536624a1027; no voters:
I20260809 06:39:57.024369 11411 leader_election.cc:288] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:39:57.024441 11413 raft_consensus.cc:2781] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:39:57.024864 11411 ts_tablet_manager.cc:1428] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 06:39:57.024977 11413 raft_consensus.cc:686] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [term 1 LEADER]: Becoming Leader. State: Replica: 8f4f2c9d8f334254a624a536624a1027, State: Running, Role: LEADER
I20260809 06:39:57.025240 11413 consensus_queue.cc:237] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } }
I20260809 06:39:57.033941 11112 catalog_manager.cc:5495] T 3e41021de9d84d32a3cf95972e93461c P 8f4f2c9d8f334254a624a536624a1027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1). New cstate: current_term: 1 leader_uuid: "8f4f2c9d8f334254a624a536624a1027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4f2c9d8f334254a624a536624a1027" member_type: VOTER last_known_addr { host: "127.6.111.1" port: 43773 } health_report { overall_health: HEALTHY } } }
I20260809 06:39:57.071511 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45450:
name: "Table.TestRecoverDeadMasterSysCatalogCopy/0.137"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:39:57.083155 11297 tablet_service.cc:1467] Processing CreateTablet for tablet 5a8cf8106d69424ba2a12c0f0343cf8c (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.137 [id=ed4c9623ffbc4d34b29f30f10f9fa120]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:39:57.083717 11297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a8cf8106d69424ba2a12c0f0343cf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:39:57.090337 11411 tablet_bootstrap.cc:492] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027: Bootstrap starting.
I20260809 06:39:57.091714 11411 tablet_bootstrap.cc:654] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027: Neither blocks nor log segments found. Creating new log.
W20260809 06:39:57.093892 11411 log.cc:1242] Error closing log: Illegal state: Log not open. State: 0
E20260809 06:39:57.095115 11411 ts_tablet_manager.cc:1419] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027: Tablet failed to bootstrap: IO error: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0/minicluster-data/ts-0/wal/wals/5a8cf8106d69424ba2a12c0f0343cf8c/.kudutmp.newsegmentJKrB9u (523231232 bytes available vs 518351011 bytes reserved) (error 28)
I20260809 06:39:57.095233 11411 ts_tablet_manager.cc:1397] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:39:57.095343 11411 tablet_replica.cc:330] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027: stopping tablet replica
I20260809 06:39:57.095441 11411 raft_consensus.cc:2229] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 06:39:57.095551 11411 raft_consensus.cc:2258] T 5a8cf8106d69424ba2a12c0f0343cf8c P 8f4f2c9d8f334254a624a536624a1027 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 06:39:57.217974 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260809 06:39:57.226765 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.006s sys 0.001s Metrics: {"bytes_written":5589,"cfile_init":1,"dirs.queue_time_us":64,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":1372,"drs_written":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4,"lbm_write_time_us":615,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260809 06:39:57.227473 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 424 bytes on disk
I20260809 06:39:57.228003 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260809 06:39:57.228528 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.195389
I20260809 06:39:57.230870 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.000083
I20260809 06:39:57.239872 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.000s Metrics: {"bytes_written":5589,"cfile_init":1,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":237,"dirs.run_wall_time_us":1197,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":628,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4}
I20260809 06:39:57.240738 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 424 bytes on disk
I20260809 06:39:57.241423 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260809 06:39:57.242095 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.195371
I20260809 06:39:57.274109 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.045s sys 0.000s Metrics: {"bytes_written":65853,"cfile_cache_hit":13,"cfile_cache_hit_bytes":51965,"cfile_cache_miss":33,"cfile_cache_miss_bytes":77353,"cfile_init":7,"delta_iterators_relevant":6,"dirs.queue_time_us":671,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1193,"drs_written":1,"lbm_read_time_us":1038,"lbm_reads_lt_1ms":61,"lbm_write_time_us":1131,"lbm_writes_lt_1ms":30,"mutex_wait_us":137,"num_input_rowsets":2,"peak_mem_usage":37327,"rows_written":277}
I20260809 06:39:57.275112 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 12115 bytes on disk
I20260809 06:39:57.275600 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20260809 06:39:57.277762 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.035s sys 0.000s Metrics: {"bytes_written":65853,"cfile_cache_hit":13,"cfile_cache_hit_bytes":51965,"cfile_cache_miss":33,"cfile_cache_miss_bytes":77465,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":850,"dirs.run_cpu_time_us":148,"dirs.run_wall_time_us":877,"drs_written":1,"lbm_read_time_us":1357,"lbm_reads_lt_1ms":65,"lbm_write_time_us":1181,"lbm_writes_lt_1ms":30,"mutex_wait_us":134,"num_input_rowsets":2,"peak_mem_usage":37532,"rows_written":277,"spinlock_wait_cycles":14976}
I20260809 06:39:57.278501 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 12115 bytes on disk
I20260809 06:39:57.278983 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:403: Failure
Failed
Bad status: Timed out: Timed out waiting for Table Creation
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1111: Failure
Expected: StartClusterWithSysCatalogGCed( &master_hps, {"--consensus_rpc_timeout_ms=2000", "--follower_unavailable_considered_failed_sec=4"}) doesn't generate new fatal failures in the current thread.
Actual: it does.
I20260809 06:40:27.007910 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 11212
I20260809 06:40:27.021997 11373 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 06:40:27.120182 11183 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Tablet 5a8cf8106d69424ba2a12c0f0343cf8c (table Table.TestRecoverDeadMasterSysCatalogCopy/0.137 [id=ed4c9623ffbc4d34b29f30f10f9fa120]) was not created within the allowed timeout. Replacing with a new tablet 00fba744111841fc886abcdea68635a8
I20260809 06:40:27.130513 11183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28f4c5713cbd4d2d96252086ef806006: Sending DeleteTablet for 1 replicas of tablet 5a8cf8106d69424ba2a12c0f0343cf8c
I20260809 06:40:27.321628 11297 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a8cf8106d69424ba2a12c0f0343cf8c with delete_type TABLET_DATA_DELETED (Replaced by 00fba744111841fc886abcdea68635a8 at 2026-08-09 06:40:27 UTC) from {username='slave'} at 127.0.0.1:58824
I20260809 06:40:27.321619 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 00fba744111841fc886abcdea68635a8 (DEFAULT_TABLE table=Table.TestRecoverDeadMasterSysCatalogCopy/0.137 [id=ed4c9623ffbc4d34b29f30f10f9fa120]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:40:27.322206 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 11212
I20260809 06:40:27.325644 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008357
I20260809 06:40:27.325629 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.008361
I20260809 06:40:27.335271 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"bytes_written":4999,"cfile_init":1,"dirs.queue_time_us":408,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":1091,"drs_written":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4,"lbm_write_time_us":637,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":293,"threads_started":1}
I20260809 06:40:27.336045 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 06:40:27.336545 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260809 06:40:27.336741 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.007s sys 0.000s Metrics: {"bytes_written":4999,"cfile_init":1,"dirs.queue_time_us":448,"dirs.run_cpu_time_us":207,"dirs.run_wall_time_us":701,"drs_written":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4,"lbm_write_time_us":864,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":286,"threads_started":1}
I20260809 06:40:27.337172 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014978
I20260809 06:40:27.337520 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 06:40:27.338034 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260809 06:40:27.338583 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014978
I20260809 06:40:27.364946 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.027s sys 0.000s Metrics: {"bytes_written":66136,"cfile_cache_hit":12,"cfile_cache_hit_bytes":52184,"cfile_cache_miss":30,"cfile_cache_miss_bytes":77068,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1663,"dirs.run_cpu_time_us":1113,"dirs.run_wall_time_us":4938,"drs_written":1,"lbm_read_time_us":1000,"lbm_reads_lt_1ms":58,"lbm_write_time_us":909,"lbm_writes_lt_1ms":30,"mutex_wait_us":103,"num_input_rowsets":2,"peak_mem_usage":37837,"rows_written":278,"thread_start_us":2171,"threads_started":3}
I20260809 06:40:27.365855 11004 maintenance_manager.cc:419] P 5d3ee6fced0d4ee2ba409f58ea250645: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 12129 bytes on disk
I20260809 06:40:27.366397 11003 maintenance_manager.cc:643] P 5d3ee6fced0d4ee2ba409f58ea250645: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260809 06:40:27.388406 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.049s sys 0.000s Metrics: {"bytes_written":66136,"cfile_cache_hit":12,"cfile_cache_hit_bytes":52184,"cfile_cache_miss":30,"cfile_cache_miss_bytes":77068,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":949,"dirs.run_cpu_time_us":813,"dirs.run_wall_time_us":3338,"drs_written":1,"lbm_read_time_us":1041,"lbm_reads_lt_1ms":58,"lbm_write_time_us":1152,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":37837,"rows_written":278,"thread_start_us":958,"threads_started":3}
I20260809 06:40:27.389374 11098 maintenance_manager.cc:419] P 28f4c5713cbd4d2d96252086ef806006: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 12129 bytes on disk
I20260809 06:40:27.389966 11097 maintenance_manager.cc:643] P 28f4c5713cbd4d2d96252086ef806006: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
W20260809 06:40:27.393133 11092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.111.1:43773 (error 108)
I20260809 06:40:27.394022 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 10978
I20260809 06:40:27.395196 11059 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 06:40:27.395203 11092 catalog_manager.cc:4584] TS 8f4f2c9d8f334254a624a536624a1027 (127.6.111.1:43773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a8cf8106d69424ba2a12c0f0343cf8c: Network error: Client connection negotiation failed: client connection to 127.6.111.1:43773: connect: Connection refused (error 111)
I20260809 06:40:27.592476 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 10978
I20260809 06:40:27.638662 6588 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_4zeaO/build/asan/bin/kudu pid 11072
I20260809 06:40:27.639777 11154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:40:27.854964 6588 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_4zeaO/build/asan/bin/kudu with pid 11072
2026-08-09T06:40:27Z chronyd exiting
I20260809 06:40:27.917037 6588 test_util.cc:175] -----------------------------------------------
I20260809 06:40:27.917166 6588 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy_0.1786257541818990-6588-0
[ FAILED ] ParameterizedRecoverMasterTest.TestRecoverDeadMasterSysCatalogCopy/0, where GetParam() = 2 (42357 ms)
[----------] 1 test from ParameterizedRecoverMasterTest (42357 ms total)
[----------] 1 test from ParameterizedAutoAddMasterTest
[ RUN ] ParameterizedAutoAddMasterTest.TestBasicAddition/2_orig_masters_not_secure
2026-08-09T06:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:40:27Z Disabled control of system clock
I20260809 06:40:27.968039 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:35343
--webserver_interface=127.6.111.62
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:35343,127.6.111.61:39273 with env {}
W20260809 06:40:28.373881 13095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:40:28.374310 13095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:40:28.374452 13095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:40:28.384145 13095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:40:28.384306 13095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:40:28.384387 13095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:40:28.384496 13095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:40:28.394459 13095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:35343,127.6.111.61:39273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.62:35343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.111.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:40:28.394929 13095 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:35343,127.6.111.61:39273'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:40:28.396222 13095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:40:28.398268 13095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:40:28.410797 13103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:40:28.410864 13101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:40:28.410769 13105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:40:28.411777 13095 server_base.cc:1034] running on GCE node
I20260809 06:40:28.412663 13095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:40:28.414181 13095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:40:28.415385 13095 hybrid_clock.cc:648] HybridClock initialized: now 1786257628415337 us; error 35 us; skew 500 ppm
I20260809 06:40:28.415769 13095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:40:28.418107 13095 webserver.cc:460] Webserver started at http://127.6.111.62:43957/ using document root <none> and password file <none>
I20260809 06:40:28.418809 13095 fs_manager.cc:359] Metadata directory not provided
I20260809 06:40:28.418924 13095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:40:28.419268 13095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:40:28.422045 13095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data/instance:
uuid: "6659aa7cd172434180736739d0705be7"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.422845 13095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal/instance:
uuid: "6659aa7cd172434180736739d0705be7"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.428149 13095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 06:40:28.431418 13112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:40:28.432667 13095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:40:28.432857 13095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal
uuid: "6659aa7cd172434180736739d0705be7"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.433099 13095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:40:28.452520 13095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:40:28.453404 13095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:40:28.453706 13095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:40:28.472831 13095 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.62:35343
I20260809 06:40:28.472865 13195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.62:35343 every 8 connection(s)
I20260809 06:40:28.474234 13095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/data/info.pb
I20260809 06:40:28.477583 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 13095
I20260809 06:40:28.477818 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-0/wal/instance
I20260809 06:40:28.478168 13196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:40:28.479079 6588 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_4zeaO/build/asan/bin/kudu
/tmp/dist-test-task_4zeaO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:39273
--webserver_interface=127.6.111.61
--webserver_port=0
--builtin_ntp_servers=127.6.111.20:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.111.62:35343,127.6.111.61:39273 with env {}
I20260809 06:40:28.484355 13196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:40:28.501708 13196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
W20260809 06:40:28.503065 13113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
W20260809 06:40:28.504791 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:28.554253 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 1
W20260809 06:40:28.556367 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:28.624751 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 2
W20260809 06:40:28.626852 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:28.718230 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 3
W20260809 06:40:28.720309 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
W20260809 06:40:28.835615 13203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:40:28.835989 13203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:40:28.836135 13203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:40:28.845053 13203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:40:28.845155 13203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:40:28.845197 13203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:40:28.845252 13203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:40:28.854405 13203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.111.20:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.111.62:35343,127.6.111.61:39273
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.111.61:39273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.111.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:54:04 UTC on ccycloud.quasar-xkgyaq.root.comops.site
build id 4575853
ASAN enabled
W20260809 06:40:28.854727 13203 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.111.62:35343,127.6.111.61:39273'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:40:28.855777 13203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:40:28.857584 13203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 06:40:28.863607 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 4
W20260809 06:40:28.865717 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
W20260809 06:40:28.870210 13210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:40:28.870062 13216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:40:28.870240 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:40:28.871172 13203 server_base.cc:1034] running on GCE node
I20260809 06:40:28.871956 13203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:40:28.873301 13203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:40:28.874569 13203 hybrid_clock.cc:648] HybridClock initialized: now 1786257628874517 us; error 42 us; skew 500 ppm
I20260809 06:40:28.874917 13203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:40:28.877132 13203 webserver.cc:460] Webserver started at http://127.6.111.61:40337/ using document root <none> and password file <none>
I20260809 06:40:28.877903 13203 fs_manager.cc:359] Metadata directory not provided
I20260809 06:40:28.878041 13203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:40:28.878368 13203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:40:28.881079 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data/instance:
uuid: "d042f6e219cc49c2a029b76c93d8a890"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.882005 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal/instance:
uuid: "d042f6e219cc49c2a029b76c93d8a890"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.887061 13203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:40:28.890568 13223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:40:28.891929 13203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:40:28.892117 13203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data,/tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal
uuid: "d042f6e219cc49c2a029b76c93d8a890"
format_stamp: "Formatted at 2026-08-09 06:40:28 on dist-test-slave-bw88"
I20260809 06:40:28.892349 13203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:40:28.907874 13203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:40:28.908754 13203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:40:28.909039 13203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:40:28.928112 13203 rpc_server.cc:307] RPC server started. Bound to: 127.6.111.61:39273
I20260809 06:40:28.928129 13296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.111.61:39273 every 8 connection(s)
I20260809 06:40:28.929661 13203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/data/info.pb
I20260809 06:40:28.933827 13298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:40:28.933818 6588 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_4zeaO/build/asan/bin/kudu as pid 13203
I20260809 06:40:28.933977 6588 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal/instance
I20260809 06:40:28.942948 13298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.62" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:40:28.954645 13298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:40:28.965065 13298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890: Bootstrap starting.
I20260809 06:40:28.968725 13298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890: Neither blocks nor log segments found. Creating new log.
I20260809 06:40:28.969992 13298 log.cc:824] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890: Log is configured to *not* fsync() on all Append() calls
W20260809 06:40:28.971349 13298 log.cc:1242] Error closing log: Illegal state: Log not open. State: 0
I20260809 06:40:28.972355 13298 tablet_replica.cc:330] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890: stopping tablet replica
I20260809 06:40:28.972606 13298 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 06:40:28.972815 13298 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d042f6e219cc49c2a029b76c93d8a890 [term 0 FOLLOWER]: Raft consensus is shut down!
E20260809 06:40:28.973520 13298 master.cc:392] Unable to init master catalog manager: IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmentbyfhjj (519864320 bytes available vs 518351011 bytes reserved) (error 28)
I20260809 06:40:28.973779 13203 master.cc:537] Master@127.6.111.61:39273 shutting down...
I20260809 06:40:28.996984 13203 master.cc:552] Master@127.6.111.61:39273 shutdown complete.
IO error: Unable to initialize catalog manager: Failed to initialize sys tables async: failed to bootstrap system catalog: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-task_4zeaO/test-tmp/dynamic_multi_master-test.0.ParameterizedAutoAddMasterTest.TestBasicAddition_2_orig_masters_not_secure.1786257541818990-6588-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000/.kudutmp.newsegmentbyfhjj (519864320 bytes available vs 518351011 bytes reserved) (error 28)
I20260809 06:40:29.165031 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 5
W20260809 06:40:29.167388 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:29.714798 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 6
W20260809 06:40:29.717433 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:30.777716 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 7
W20260809 06:40:30.780339 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
I20260809 06:40:32.870669 13196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.111.61" port: 39273 } attempt: 8
W20260809 06:40:32.873246 13196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.111.61:39273: Network error: Client connection negotiation failed: client connection to 127.6.111.61:39273: connect: Connection refused (error 111)
F20260809 06:40:33.966594 6588 external_mini_cluster.cc:634] Check failed: leader_master() != nullptr Must have started at least 1 master before adding tablet servers
*** Check failure stack trace: ***
*** Aborted at 1786257633 (unix time) try "date -d @1786257633" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000019bc) received by PID 6588 (TID 0x7fad13fb9d40) from PID 6588; stack trace: ***
@ 0x7fad17705a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fad1f755980 (unknown) at ??:0
@ 0x7fad14ca9e87 gsignal at ??:0
@ 0x7fad14cab7f1 abort at ??:0
@ 0x7fad176f79ad google::LogMessage::Fail() at ??:0
@ 0x7fad176fa88a google::LogMessage::SendToLog() at ??:0
@ 0x7fad176f74de google::LogMessage::Flush() at ??:0
@ 0x7fad176f8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fad2d14bbc5 kudu::cluster::ExternalMiniCluster::AddTabletServer() at ??:0
@ 0x7fad2d1453f0 kudu::cluster::ExternalMiniCluster::Start() at ??:0
@ 0x55506a kudu::master::AutoAddMasterTest::SetUpWithTestArgs() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1466
@ 0x555aa1 kudu::master::ParameterizedAutoAddMasterTest::SetUp() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/master/dynamic_multi_master-test.cc:1824
@ 0x7fad19514bfe testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7fad19509566 testing::Test::Run() at ??:0
@ 0x7fad19509795 testing::TestInfo::Run() at ??:0
@ 0x7fad19509cdf testing::TestSuite::Run() at ??:0
@ 0x7fad1950a29f testing::internal::UnitTestImpl::RunAllTests() at ??:0
@ 0x7fad1951513e testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7fad1950983d testing::UnitTest::Run() at ??:0
@ 0x5d3fa2 RUN_ALL_TESTS() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/gtest/gtest.h:2293
@ 0x5d24b2 main at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:109
@ 0x7fad14c8cc87 __libc_start_main at ??:0
@ 0x413329 (unknown) at ??:?